[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:58 dist-test-slave-7vjv krb5kdc[2367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:58 dist-test-slave-7vjv krb5kdc[2367](info): set up 2 sockets
Aug 09 23:13:58 dist-test-slave-7vjv krb5kdc[2367](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 23:13:59.776379 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.455s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260809 23:13:59.969744 2355 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.193s user 0.002s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260809 23:14:00.122694 2355 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.153s user 0.001s sys 0.001s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260809 23:14:00.269550 2355 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.147s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:14:00 dist-test-slave-7vjv krb5kdc[2367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:14:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:14:00.322629 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:36635
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:36635
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:00.401489 2384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:00.401695 2384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:00.401731 2384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:00.402765 2384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:00.402837 2384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:00.402858 2384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:00.402876 2384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:00.402904 2384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:14:00.404366 2384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:36635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:36635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:00.404740 2384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:00.405012 2384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:00.411839 2397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:00.411753 2384 server_base.cc:1034] running on GCE node
W20260809 23:14:00.412858 2405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:00.413110 2395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:00.413400 2384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:00.413806 2384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:00.414963 2384 hybrid_clock.cc:648] HybridClock initialized: now 1786317240414944 us; error 63 us; skew 500 ppm
Aug 09 23:14:00 dist-test-slave-7vjv krb5kdc[2367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317240, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:00.417433 2384 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:14:00.418090 2384 webserver.cc:460] Webserver started at http://127.2.76.254:37327/ using document root <none> and password file <none>
I20260809 23:14:00.418375 2384 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:00.418452 2384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:00.418650 2384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:00.419826 2384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "a00f27af8043493c8998d0541b149bab"
format_stamp: "Formatted at 2026-08-09 23:14:00 on dist-test-slave-7vjv"
server_key: "7da005bb60fab1028532930e4b7ed327"
server_key_iv: "0417771371f31a8629c7559a22b6e5da"
server_key_version: "encryptionkey@0"
I20260809 23:14:00.420411 2384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "a00f27af8043493c8998d0541b149bab"
format_stamp: "Formatted at 2026-08-09 23:14:00 on dist-test-slave-7vjv"
server_key: "7da005bb60fab1028532930e4b7ed327"
server_key_iv: "0417771371f31a8629c7559a22b6e5da"
server_key_version: "encryptionkey@0"
I20260809 23:14:00.422421 2384 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 23:14:00.423414 2419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:00.423682 2384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:14:00.423791 2384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "a00f27af8043493c8998d0541b149bab"
format_stamp: "Formatted at 2026-08-09 23:14:00 on dist-test-slave-7vjv"
server_key: "7da005bb60fab1028532930e4b7ed327"
server_key_iv: "0417771371f31a8629c7559a22b6e5da"
server_key_version: "encryptionkey@0"
I20260809 23:14:00.423913 2384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:00.434110 2384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:00.436877 2384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:00.437104 2384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:00.443274 2384 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:36635
I20260809 23:14:00.443343 2504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:36635 every 8 connection(s)
I20260809 23:14:00.443848 2384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:00.445165 2505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:00.446746 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 2384
I20260809 23:14:00.446887 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:00.447803 2355 external_mini_cluster.cc:1428] Setting key 578a2f914ad09b28af18b9246154f90d
I20260809 23:14:00.449254 2505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab: Bootstrap starting.
I20260809 23:14:00.450366 2505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:00.450733 2505 log.cc:824] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:00.451609 2505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab: No bootstrap required, opened a new log
I20260809 23:14:00.454447 2505 raft_consensus.cc:348] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00f27af8043493c8998d0541b149bab" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36635 } }
I20260809 23:14:00.454694 2505 raft_consensus.cc:374] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:00.454797 2505 raft_consensus.cc:729] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a00f27af8043493c8998d0541b149bab, State: Initialized, Role: FOLLOWER
I20260809 23:14:00.455018 2505 consensus_queue.cc:260] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00f27af8043493c8998d0541b149bab" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36635 } }
I20260809 23:14:00.455168 2505 raft_consensus.cc:388] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:00.455221 2505 raft_consensus.cc:482] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:00.455273 2505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:00.456233 2505 raft_consensus.cc:504] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00f27af8043493c8998d0541b149bab" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36635 } }
I20260809 23:14:00.456409 2505 leader_election.cc:302] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a00f27af8043493c8998d0541b149bab; no voters:
I20260809 23:14:00.456671 2505 leader_election.cc:288] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:00.456707 2518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:00.456871 2518 raft_consensus.cc:686] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [term 1 LEADER]: Becoming Leader. State: Replica: a00f27af8043493c8998d0541b149bab, State: Running, Role: LEADER
I20260809 23:14:00.457089 2518 consensus_queue.cc:237] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00f27af8043493c8998d0541b149bab" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36635 } }
I20260809 23:14:00.457180 2505 sys_catalog.cc:564] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:00.457676 2523 sys_catalog.cc:455] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a00f27af8043493c8998d0541b149bab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00f27af8043493c8998d0541b149bab" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36635 } } }
I20260809 23:14:00.457886 2523 sys_catalog.cc:458] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:00.458549 2522 sys_catalog.cc:455] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [sys.catalog]: SysCatalogTable state changed. Reason: New leader a00f27af8043493c8998d0541b149bab. Latest consensus state: current_term: 1 leader_uuid: "a00f27af8043493c8998d0541b149bab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a00f27af8043493c8998d0541b149bab" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36635 } } }
I20260809 23:14:00.458676 2522 sys_catalog.cc:458] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:00.458714 2532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:00.459913 2532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:00.462474 2532 catalog_manager.cc:1347] Generated new cluster ID: 028037a323384ccdb5ef36c866610363
I20260809 23:14:00.462527 2532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:14:00 dist-test-slave-7vjv krb5kdc[2367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:00.469257 2513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:00.449414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36204 (local address 127.2.76.254:36635)
0809 23:14:00.449713 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:00.449723 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:00.451991 (+ 2268us) server_negotiation.cc:407] Connection header received
0809 23:14:00.454644 (+ 2653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:00.454649 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:00.454718 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:00.454868 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:00.456356 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.457495 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.458995 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.459625 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.459770 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:00.464912 (+ 5142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:00.464916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:00.464917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:00.464928 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:00.467048 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:00.467549 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:00.467552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:00.467553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:00.467704 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:00.468021 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:00.468022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:00.468022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:00.468361 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:00.468524 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:00.468805 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 23:14:00.468997 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":77,"threads_started":1}
I20260809 23:14:00.472169 2532 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:00.472877 2532 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:00.483361 2532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a00f27af8043493c8998d0541b149bab: Generated new TSK 0
I20260809 23:14:00.483508 2532 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:14:00.510421 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36635
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:00.586745 2556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:00.586958 2556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:00.586993 2556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:00.588001 2556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:00.588059 2556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:00.588099 2556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:14:00.589505 2556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36635
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:00.589824 2556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:00.590047 2556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:00.595610 2573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:00.595548 2569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:00.595534 2571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:00.595945 2556 server_base.cc:1034] running on GCE node
I20260809 23:14:00.596254 2556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:00.596521 2556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:00.597720 2556 hybrid_clock.cc:648] HybridClock initialized: now 1786317240597707 us; error 41 us; skew 500 ppm
Aug 09 23:14:00 dist-test-slave-7vjv krb5kdc[2367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317240, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:00.600706 2556 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:14:00.601257 2556 webserver.cc:460] Webserver started at http://127.2.76.193:46333/ using document root <none> and password file <none>
I20260809 23:14:00.601490 2556 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:00.601552 2556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:00.601692 2556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:00.602589 2556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "71d34276b2054521a561706d8ed51b6c"
format_stamp: "Formatted at 2026-08-09 23:14:00 on dist-test-slave-7vjv"
server_key: "67b5ab578cbb361d612a34225c73645d"
server_key_iv: "3c84b2810c9570cebec4bcb3e7cb0be2"
server_key_version: "encryptionkey@0"
I20260809 23:14:00.602952 2556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "71d34276b2054521a561706d8ed51b6c"
format_stamp: "Formatted at 2026-08-09 23:14:00 on dist-test-slave-7vjv"
server_key: "67b5ab578cbb361d612a34225c73645d"
server_key_iv: "3c84b2810c9570cebec4bcb3e7cb0be2"
server_key_version: "encryptionkey@0"
I20260809 23:14:00.604354 2556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:14:00.605150 2585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:00.605326 2556 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:00.605430 2556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "71d34276b2054521a561706d8ed51b6c"
format_stamp: "Formatted at 2026-08-09 23:14:00 on dist-test-slave-7vjv"
server_key: "67b5ab578cbb361d612a34225c73645d"
server_key_iv: "3c84b2810c9570cebec4bcb3e7cb0be2"
server_key_version: "encryptionkey@0"
I20260809 23:14:00.605501 2556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:00.612146 2556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:00.614114 2556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:00.614256 2556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:00.614535 2556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:00.614853 2556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:00.614938 2556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:00.614974 2556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:00.614998 2556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:00.624711 2556 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:33961
I20260809 23:14:00.624773 2783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:33961 every 8 connection(s)
I20260809 23:14:00.625075 2556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:14:00.625365 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 2556
I20260809 23:14:00.625474 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:14:00.625641 2355 external_mini_cluster.cc:1428] Setting key 4d9f817da6911c374b001e0876594e77
Aug 09 23:14:00 dist-test-slave-7vjv krb5kdc[2367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317240, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:00.635177 2513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:00.625981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:59791 (local address 127.2.76.254:36635)
0809 23:14:00.626018 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:00.626020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:00.626279 (+ 259us) server_negotiation.cc:407] Connection header received
0809 23:14:00.626879 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:00.626880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:00.626894 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:00.626938 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:00.628049 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.628583 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.629576 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.629896 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.629960 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:00.632256 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:00.632257 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:00.632257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:00.632264 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:00.633565 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:00.634148 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:00.634149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:00.634149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:00.634210 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:00.634546 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:00.634546 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:00.634547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:00.634649 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:00.634696 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:00.635048 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 23:14:00.635115 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 23:14:00.635411 2786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:00.625979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:36635 (local address 127.2.76.193:59791)
0809 23:14:00.626170 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:00.626201 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:00.626776 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:00.626993 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:00.626996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:00.627753 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:00.627941 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:00.627946 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.628683 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.628684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:00.629478 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:00.629480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.629970 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.629971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:00.630066 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:00.630515 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:00.630522 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:00.632167 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:00.633646 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:00.633654 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:00.633656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:00.633961 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:00.634322 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:00.634323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:00.634323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:00.634465 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:00.634714 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:00.634721 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:00.634898 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 23:14:00.635029 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 23:14:00.635209 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
I20260809 23:14:00.636004 2784 heartbeater.cc:347] Connected to a master server at 127.2.76.254:36635
I20260809 23:14:00.636123 2784 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:00.636289 2784 heartbeater.cc:511] Master 127.2.76.254:36635 requested a full tablet report, sending...
I20260809 23:14:00.637082 2445 ts_manager.cc:194] Registered new tserver with Master: 71d34276b2054521a561706d8ed51b6c (127.2.76.193:33961)
I20260809 23:14:00.637696 2445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:59791
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:14:00.652750 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36635
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:00.730710 2795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:00.730868 2795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:00.730888 2795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:00.731904 2795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:00.731943 2795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:00.731992 2795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:14:00.733394 2795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36635
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:00.733600 2795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:00.733810 2795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:00.738034 2807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:00.738044 2804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:00.738130 2803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:00.738458 2795 server_base.cc:1034] running on GCE node
I20260809 23:14:00.738641 2795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:00.738864 2795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:00.740017 2795 hybrid_clock.cc:648] HybridClock initialized: now 1786317240740002 us; error 26 us; skew 500 ppm
Aug 09 23:14:00 dist-test-slave-7vjv krb5kdc[2367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317240, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:00.744766 2795 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:14:00.745183 2795 webserver.cc:460] Webserver started at http://127.2.76.194:35367/ using document root <none> and password file <none>
I20260809 23:14:00.745373 2795 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:00.745412 2795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:00.745545 2795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:00.746485 2795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "ab6541b0f2094ff79950c217c3ef713f"
format_stamp: "Formatted at 2026-08-09 23:14:00 on dist-test-slave-7vjv"
server_key: "f8d490599e3897b563deff0342fbdfee"
server_key_iv: "1393721991d0ef42ec5402e7dd310767"
server_key_version: "encryptionkey@0"
I20260809 23:14:00.746861 2795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "ab6541b0f2094ff79950c217c3ef713f"
format_stamp: "Formatted at 2026-08-09 23:14:00 on dist-test-slave-7vjv"
server_key: "f8d490599e3897b563deff0342fbdfee"
server_key_iv: "1393721991d0ef42ec5402e7dd310767"
server_key_version: "encryptionkey@0"
I20260809 23:14:00.748188 2795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 23:14:00.748823 2817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:00.748976 2795 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:00.749042 2795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "ab6541b0f2094ff79950c217c3ef713f"
format_stamp: "Formatted at 2026-08-09 23:14:00 on dist-test-slave-7vjv"
server_key: "f8d490599e3897b563deff0342fbdfee"
server_key_iv: "1393721991d0ef42ec5402e7dd310767"
server_key_version: "encryptionkey@0"
I20260809 23:14:00.749120 2795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:00.766768 2795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:00.768373 2795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:00.768527 2795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:00.768718 2795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:00.769083 2795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:00.769117 2795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:00.769138 2795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:00.769204 2795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:00.778120 2795 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:38509
I20260809 23:14:00.778182 2997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:38509 every 8 connection(s)
I20260809 23:14:00.778471 2795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:14:00 dist-test-slave-7vjv krb5kdc[2367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317240, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:00.786749 2513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:00.779201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:44269 (local address 127.2.76.254:36635)
0809 23:14:00.779251 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:00.779253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:00.779579 (+ 326us) server_negotiation.cc:407] Connection header received
0809 23:14:00.780010 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:00.780011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:00.780021 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:00.780072 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:00.780975 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.781473 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.782357 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.782537 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.782585 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:00.784532 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:00.784533 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:00.784534 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:00.784540 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:00.785689 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:00.786018 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:00.786018 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:00.786019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:00.786067 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:00.786262 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:00.786263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:00.786263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:00.786379 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:00.786428 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:00.786633 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 23:14:00.786691 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:14:00.786892 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:00.779112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:36635 (local address 127.2.76.194:44269)
0809 23:14:00.779439 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:00.779485 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:00.779927 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:00.780179 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:00.780182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:00.780727 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:00.780897 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:00.780902 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.781551 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.781552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:00.782285 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:00.782287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.782608 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.782609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:00.782698 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:00.783192 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:00.783200 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:00.784449 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:00.785756 (+ 1307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:00.785763 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:00.785764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:00.785954 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:00.786122 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:00.786123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:00.786123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:00.786212 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:00.786431 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:00.786435 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:00.786533 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 23:14:00.786606 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:14:00.786707 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":170,"threads_started":1}
I20260809 23:14:00.787297 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 2795
I20260809 23:14:00.787400 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:14:00.787405 2998 heartbeater.cc:347] Connected to a master server at 127.2.76.254:36635
I20260809 23:14:00.787496 2998 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:00.787556 2355 external_mini_cluster.cc:1428] Setting key d2feba73b412bd9f49f4d52968d1f5c4
I20260809 23:14:00.787655 2998 heartbeater.cc:511] Master 127.2.76.254:36635 requested a full tablet report, sending...
I20260809 23:14:00.788056 2444 ts_manager.cc:194] Registered new tserver with Master: ab6541b0f2094ff79950c217c3ef713f (127.2.76.194:38509)
I20260809 23:14:00.788435 2444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:44269
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:14:00.812747 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36635
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:00.888823 3011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:00.888988 3011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:00.889008 3011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:00.890013 3011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:00.890046 3011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:00.890071 3011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:14:00.891613 3011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36635
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:00.891790 3011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:00.891973 3011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:00.896875 3017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:00.896895 3016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:00.896886 3025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:00.897080 3011 server_base.cc:1034] running on GCE node
I20260809 23:14:00.897277 3011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:00.897554 3011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:00.898705 3011 hybrid_clock.cc:648] HybridClock initialized: now 1786317240898693 us; error 22 us; skew 500 ppm
Aug 09 23:14:00 dist-test-slave-7vjv krb5kdc[2367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317240, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:00.901122 3011 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:14:00.901547 3011 webserver.cc:460] Webserver started at http://127.2.76.195:35297/ using document root <none> and password file <none>
I20260809 23:14:00.901747 3011 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:00.901808 3011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:00.901947 3011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:00.902994 3011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "6e647c66a811445c920d9abe006aba08"
format_stamp: "Formatted at 2026-08-09 23:14:00 on dist-test-slave-7vjv"
server_key: "0dd4ae4c19f6877d7a875516c0d70c4a"
server_key_iv: "caffc496a44b590ef39cecd35cb5acc3"
server_key_version: "encryptionkey@0"
I20260809 23:14:00.903393 3011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "6e647c66a811445c920d9abe006aba08"
format_stamp: "Formatted at 2026-08-09 23:14:00 on dist-test-slave-7vjv"
server_key: "0dd4ae4c19f6877d7a875516c0d70c4a"
server_key_iv: "caffc496a44b590ef39cecd35cb5acc3"
server_key_version: "encryptionkey@0"
I20260809 23:14:00.904762 3011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:14:00.905483 3035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:00.905652 3011 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:00.905753 3011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "6e647c66a811445c920d9abe006aba08"
format_stamp: "Formatted at 2026-08-09 23:14:00 on dist-test-slave-7vjv"
server_key: "0dd4ae4c19f6877d7a875516c0d70c4a"
server_key_iv: "caffc496a44b590ef39cecd35cb5acc3"
server_key_version: "encryptionkey@0"
I20260809 23:14:00.905824 3011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:00.927587 3011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:00.929374 3011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:00.929534 3011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:00.929831 3011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:00.930166 3011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:00.930217 3011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:00.930253 3011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:00.930277 3011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:00.937983 3011 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:34719
I20260809 23:14:00.938016 3252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:34719 every 8 connection(s)
I20260809 23:14:00.938282 3011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:14:00 dist-test-slave-7vjv krb5kdc[2367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317240, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:00.946157 2513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:00.939043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:52953 (local address 127.2.76.254:36635)
0809 23:14:00.939118 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:00.939120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:00.939320 (+ 200us) server_negotiation.cc:407] Connection header received
0809 23:14:00.939723 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:00.939724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:00.939732 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:00.939778 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:00.940602 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.941037 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.941866 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.942031 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.942076 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:00.943896 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:00.943897 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:00.943898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:00.943904 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:00.945098 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:00.945451 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:00.945452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:00.945452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:00.945531 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:00.945729 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:00.945730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:00.945730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:00.945813 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:00.945856 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:00.946048 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 23:14:00.946101 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 23:14:00.946310 3256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:00.939001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:36635 (local address 127.2.76.195:52953)
0809 23:14:00.939220 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:00.939250 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:00.939649 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:00.939822 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:00.939825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:00.940368 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:00.940534 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:00.940538 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.941111 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.941112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:00.941800 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:00.941802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.942098 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.942098 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:00.942182 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:00.942554 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:00.942561 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:00.943817 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:00.945164 (+ 1347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:00.945170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:00.945172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:00.945359 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:00.945587 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:00.945588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:00.945588 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:00.945682 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:00.945860 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:00.945864 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:00.945950 (+ 86us) client_negotiation.cc:769] Sending connection context
0809 23:14:00.946017 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 23:14:00.946111 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":103,"threads_started":1}
I20260809 23:14:00.946769 3253 heartbeater.cc:347] Connected to a master server at 127.2.76.254:36635
I20260809 23:14:00.946843 3253 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:00.947011 3253 heartbeater.cc:511] Master 127.2.76.254:36635 requested a full tablet report, sending...
I20260809 23:14:00.947399 2444 ts_manager.cc:194] Registered new tserver with Master: 6e647c66a811445c920d9abe006aba08 (127.2.76.195:34719)
I20260809 23:14:00.947489 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 3011
I20260809 23:14:00.947580 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:14:00.947731 2355 external_mini_cluster.cc:1428] Setting key 27fe846633dcad5750ad7f3ceafd2660
I20260809 23:14:00.947824 2444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:52953
I20260809 23:14:00.948765 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 23:14:00 dist-test-slave-7vjv krb5kdc[2367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317240, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 23:14:00 dist-test-slave-7vjv krb5kdc[2367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317240, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.76.193@KRBTEST.COM
I20260809 23:14:00.972442 3272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:00.965521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51178 (local address 127.2.76.193:33961)
0809 23:14:00.965697 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:00.965701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:00.965710 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:14:00.965732 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:00.965734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:00.965746 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:00.965809 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:00.966297 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.966946 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.967559 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.967830 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.967876 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:00.969875 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:00.969882 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:00.969882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:00.969888 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:00.971519 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:00.971803 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:00.971805 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:00.971805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:00.971856 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:00.971995 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:00.971995 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:00.971996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:00.972158 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:00.972261 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:00.972312 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 23:14:00.972371 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
W20260809 23:14:00.972895 2636 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51178
Aug 09 23:14:00 dist-test-slave-7vjv krb5kdc[2367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 23:14:00 dist-test-slave-7vjv krb5kdc[2367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.193@KRBTEST.COM
I20260809 23:14:00.994843 3272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:00.988762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51180 (local address 127.2.76.193:33961)
0809 23:14:00.988841 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:00.988842 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:00.988982 (+ 140us) server_negotiation.cc:407] Connection header received
0809 23:14:00.989035 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:00.989036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:00.989044 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:00.989089 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:00.989568 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.989987 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.990634 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:00.990816 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:00.990862 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:00.992809 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:00.992810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:00.992810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:00.992816 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:00.994045 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:00.994297 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:00.994298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:00.994298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:00.994343 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:00.994472 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:00.994472 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:00.994473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:00.994546 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:00.994620 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:00.994680 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 23:14:00.994732 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 23:14:00.995529 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 2556
I20260809 23:14:01.008344 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 2795
I20260809 23:14:01.023507 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 3011
I20260809 23:14:01.032073 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 2384
2026-08-09T23:14:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2739 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:14:01 dist-test-slave-7vjv krb5kdc[3282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:14:01 dist-test-slave-7vjv krb5kdc[3282](info): set up 2 sockets
Aug 09 23:14:01 dist-test-slave-7vjv krb5kdc[3282](info): commencing operation
krb5kdc: starting...
W20260809 23:14:02.214092 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:14:02 dist-test-slave-7vjv krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:14:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:14:02.316967 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45837
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:45837
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:02.399546 3303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:02.399755 3303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:02.399788 3303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:02.400836 3303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:02.400897 3303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:02.400919 3303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:02.400946 3303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:02.400969 3303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:14:02.402321 3303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:45837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:02.402575 3303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:02.402781 3303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:02.407359 3312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:02.407529 3303 server_base.cc:1034] running on GCE node
W20260809 23:14:02.407337 3310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:02.407356 3314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:02.407872 3303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:02.408195 3303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:02.409361 3303 hybrid_clock.cc:648] HybridClock initialized: now 1786317242409331 us; error 39 us; skew 500 ppm
Aug 09 23:14:02 dist-test-slave-7vjv krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317242, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:02.411628 3303 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:14:02.412135 3303 webserver.cc:460] Webserver started at http://127.2.76.254:45095/ using document root <none> and password file <none>
I20260809 23:14:02.412328 3303 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:02.412384 3303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:02.412541 3303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:02.413542 3303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "a976ae4bdb064d179bc60d4cb5199ec4"
format_stamp: "Formatted at 2026-08-09 23:14:02 on dist-test-slave-7vjv"
server_key: "db72ef6bf595931ddcebbf0332b64d94"
server_key_iv: "b93d3de7119b0edbe169ac07920a3b19"
server_key_version: "encryptionkey@0"
I20260809 23:14:02.413869 3303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "a976ae4bdb064d179bc60d4cb5199ec4"
format_stamp: "Formatted at 2026-08-09 23:14:02 on dist-test-slave-7vjv"
server_key: "db72ef6bf595931ddcebbf0332b64d94"
server_key_iv: "b93d3de7119b0edbe169ac07920a3b19"
server_key_version: "encryptionkey@0"
I20260809 23:14:02.415118 3303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:14:02.415822 3328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:02.415966 3303 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:02.416051 3303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "a976ae4bdb064d179bc60d4cb5199ec4"
format_stamp: "Formatted at 2026-08-09 23:14:02 on dist-test-slave-7vjv"
server_key: "db72ef6bf595931ddcebbf0332b64d94"
server_key_iv: "b93d3de7119b0edbe169ac07920a3b19"
server_key_version: "encryptionkey@0"
I20260809 23:14:02.416139 3303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:02.429368 3303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:02.431207 3303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:02.431309 3303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:02.435393 3303 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:45837
I20260809 23:14:02.435441 3420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:45837 every 8 connection(s)
I20260809 23:14:02.435750 3303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:02.436574 3421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:02.438552 3421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4: Bootstrap starting.
I20260809 23:14:02.439123 3421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:02.439361 3421 log.cc:824] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:02.440006 3421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4: No bootstrap required, opened a new log
I20260809 23:14:02.441530 3421 raft_consensus.cc:348] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a976ae4bdb064d179bc60d4cb5199ec4" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45837 } }
I20260809 23:14:02.441658 3421 raft_consensus.cc:374] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:02.441677 3421 raft_consensus.cc:729] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a976ae4bdb064d179bc60d4cb5199ec4, State: Initialized, Role: FOLLOWER
I20260809 23:14:02.441771 3421 consensus_queue.cc:260] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a976ae4bdb064d179bc60d4cb5199ec4" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45837 } }
I20260809 23:14:02.441845 3421 raft_consensus.cc:388] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:02.441881 3421 raft_consensus.cc:482] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:02.441947 3421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:02.442451 3421 raft_consensus.cc:504] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a976ae4bdb064d179bc60d4cb5199ec4" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45837 } }
I20260809 23:14:02.442562 3421 leader_election.cc:302] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a976ae4bdb064d179bc60d4cb5199ec4; no voters:
I20260809 23:14:02.442756 3421 leader_election.cc:288] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:02.442827 3428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:02.442919 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 3303
I20260809 23:14:02.443011 3421 sys_catalog.cc:564] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:02.443058 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:02.443022 3428 raft_consensus.cc:686] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [term 1 LEADER]: Becoming Leader. State: Replica: a976ae4bdb064d179bc60d4cb5199ec4, State: Running, Role: LEADER
I20260809 23:14:02.443245 2355 external_mini_cluster.cc:1428] Setting key f158c541dfbfb937f6c19529189c67be
I20260809 23:14:02.443272 3428 consensus_queue.cc:237] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a976ae4bdb064d179bc60d4cb5199ec4" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45837 } }
I20260809 23:14:02.444341 3430 sys_catalog.cc:455] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a976ae4bdb064d179bc60d4cb5199ec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a976ae4bdb064d179bc60d4cb5199ec4" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45837 } } }
I20260809 23:14:02.444437 3430 sys_catalog.cc:458] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:02.444705 3431 sys_catalog.cc:455] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a976ae4bdb064d179bc60d4cb5199ec4. Latest consensus state: current_term: 1 leader_uuid: "a976ae4bdb064d179bc60d4cb5199ec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a976ae4bdb064d179bc60d4cb5199ec4" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45837 } } }
I20260809 23:14:02.444976 3431 sys_catalog.cc:458] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4 [sys.catalog]: This master's current role is: LEADER
W20260809 23:14:02.445092 3454 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:14:02.445156 3454 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:14:02.445276 3455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:02.445452 3455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:02.446929 3455 catalog_manager.cc:1347] Generated new cluster ID: 1b4198d085ef413ca47687adddae8275
I20260809 23:14:02.446983 3455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:14:02 dist-test-slave-7vjv krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:02.453903 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:02.443948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55466 (local address 127.2.76.254:45837)
0809 23:14:02.444160 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:02.444168 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:02.444197 (+ 29us) server_negotiation.cc:407] Connection header received
0809 23:14:02.444298 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:02.444305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:02.444356 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:02.445055 (+ 699us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:02.445303 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.446375 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.447349 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.447761 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.447882 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:02.450099 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:02.450103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:02.450103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:02.450113 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:02.452453 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:02.452850 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:02.452853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:02.452853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:02.452941 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:02.453150 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:02.453150 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:02.453151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:02.453355 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:02.453524 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:02.453586 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 23:14:02.453715 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":92,"threads_started":1}
I20260809 23:14:02.460188 3455 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:02.460675 3455 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:02.463421 3455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a976ae4bdb064d179bc60d4cb5199ec4: Generated new TSK 0
I20260809 23:14:02.463622 3455 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:14:02.492851 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45837
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:02.580767 3466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:02.580966 3466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:02.581009 3466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:02.582059 3466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:02.582119 3466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:02.582160 3466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:14:02.583499 3466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45837
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:02.583742 3466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:02.583940 3466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:02.588724 3480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:02.588750 3477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:02.588949 3466 server_base.cc:1034] running on GCE node
W20260809 23:14:02.588750 3482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:02.589206 3466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:02.589418 3466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:02.590595 3466 hybrid_clock.cc:648] HybridClock initialized: now 1786317242590586 us; error 25 us; skew 500 ppm
Aug 09 23:14:02 dist-test-slave-7vjv krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317242, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:02.593107 3466 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:14:02.593770 3466 webserver.cc:460] Webserver started at http://127.2.76.193:38965/ using document root <none> and password file <none>
I20260809 23:14:02.594014 3466 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:02.594077 3466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:02.594203 3466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:02.595158 3466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "216698123d00495a84e51def125fe375"
format_stamp: "Formatted at 2026-08-09 23:14:02 on dist-test-slave-7vjv"
server_key: "d3f01a104cfa41f86186b11ed89fdfb9"
server_key_iv: "ba3c551a746bc8908cb96cfc2aad3c7e"
server_key_version: "encryptionkey@0"
I20260809 23:14:02.595516 3466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "216698123d00495a84e51def125fe375"
format_stamp: "Formatted at 2026-08-09 23:14:02 on dist-test-slave-7vjv"
server_key: "d3f01a104cfa41f86186b11ed89fdfb9"
server_key_iv: "ba3c551a746bc8908cb96cfc2aad3c7e"
server_key_version: "encryptionkey@0"
I20260809 23:14:02.597091 3466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:14:02.597944 3493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:02.598107 3466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:02.598209 3466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "216698123d00495a84e51def125fe375"
format_stamp: "Formatted at 2026-08-09 23:14:02 on dist-test-slave-7vjv"
server_key: "d3f01a104cfa41f86186b11ed89fdfb9"
server_key_iv: "ba3c551a746bc8908cb96cfc2aad3c7e"
server_key_version: "encryptionkey@0"
I20260809 23:14:02.598287 3466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:02.605454 3466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:02.607631 3466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:02.607774 3466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:02.607976 3466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:02.608304 3466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:02.608359 3466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:02.608398 3466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:02.608424 3466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:02.614209 3466 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:35983
I20260809 23:14:02.614272 3684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:35983 every 8 connection(s)
I20260809 23:14:02.614562 3466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:14:02.617559 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 3466
I20260809 23:14:02.617671 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:14:02.617851 2355 external_mini_cluster.cc:1428] Setting key f9da303a66d06bd24bac9b34f2b5f593
Aug 09 23:14:02 dist-test-slave-7vjv krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317242, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:02.624266 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:02.615171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:46591 (local address 127.2.76.254:45837)
0809 23:14:02.615213 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:02.615215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:02.615350 (+ 135us) server_negotiation.cc:407] Connection header received
0809 23:14:02.616427 (+ 1077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:02.616428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:02.616442 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:02.616500 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:02.617006 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.617522 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.618777 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.619091 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.619179 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:02.621615 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:02.621616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:02.621616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:02.621622 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:02.622846 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:02.623281 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:02.623282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:02.623283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:02.623357 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:02.623636 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:02.623637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:02.623637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:02.623737 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:02.623803 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:02.624046 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 23:14:02.624186 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:14:02.624363 3693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:02.615117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45837 (local address 127.2.76.193:46591)
0809 23:14:02.615250 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:02.615283 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:02.616337 (+ 1054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:02.616558 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:02.616561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:02.616762 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:02.616936 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:02.616940 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.617611 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.617612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:02.618694 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:02.618696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.619184 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.619185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:02.619284 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:02.619849 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:02.619863 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:02.621511 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:02.622951 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:02.622959 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:02.622961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:02.623198 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:02.623422 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:02.623423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:02.623423 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:02.623546 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:02.623797 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:02.623805 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:02.623927 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 23:14:02.624021 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 23:14:02.624181 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":81,"threads_started":1}
I20260809 23:14:02.624739 3686 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45837
I20260809 23:14:02.624833 3686 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:02.624994 3686 heartbeater.cc:511] Master 127.2.76.254:45837 requested a full tablet report, sending...
I20260809 23:14:02.625413 3349 ts_manager.cc:194] Registered new tserver with Master: 216698123d00495a84e51def125fe375 (127.2.76.193:35983)
I20260809 23:14:02.625908 3349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:46591
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:14:02.642679 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45837
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:02.715878 3700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:02.716048 3700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:02.716068 3700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:02.717176 3700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:02.717213 3700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:02.717236 3700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:14:02.718441 3700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45837
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:02.718638 3700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:02.718832 3700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:02.723281 3713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:02.723373 3709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:02.723306 3715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:02.723438 3700 server_base.cc:1034] running on GCE node
I20260809 23:14:02.723704 3700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:02.723887 3700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:02.725018 3700 hybrid_clock.cc:648] HybridClock initialized: now 1786317242725007 us; error 27 us; skew 500 ppm
Aug 09 23:14:02 dist-test-slave-7vjv krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317242, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:02.727246 3700 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:14:02.727658 3700 webserver.cc:460] Webserver started at http://127.2.76.194:32931/ using document root <none> and password file <none>
I20260809 23:14:02.727856 3700 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:02.727919 3700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:02.728060 3700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:02.728955 3700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "34bcded5bfa3400292fc15686f68e48c"
format_stamp: "Formatted at 2026-08-09 23:14:02 on dist-test-slave-7vjv"
server_key: "6a898b834a8fab6480fe4ef02b9bafc8"
server_key_iv: "d2b8043b1651bc7fae7f65b7625ede61"
server_key_version: "encryptionkey@0"
I20260809 23:14:02.729313 3700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "34bcded5bfa3400292fc15686f68e48c"
format_stamp: "Formatted at 2026-08-09 23:14:02 on dist-test-slave-7vjv"
server_key: "6a898b834a8fab6480fe4ef02b9bafc8"
server_key_iv: "d2b8043b1651bc7fae7f65b7625ede61"
server_key_version: "encryptionkey@0"
I20260809 23:14:02.730548 3700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:14:02.731153 3726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:02.731314 3700 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:02.731405 3700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "34bcded5bfa3400292fc15686f68e48c"
format_stamp: "Formatted at 2026-08-09 23:14:02 on dist-test-slave-7vjv"
server_key: "6a898b834a8fab6480fe4ef02b9bafc8"
server_key_iv: "d2b8043b1651bc7fae7f65b7625ede61"
server_key_version: "encryptionkey@0"
I20260809 23:14:02.731475 3700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:02.739205 3700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:02.741267 3700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:02.741420 3700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:02.741631 3700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:02.741909 3700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:02.741972 3700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:02.742017 3700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:02.742043 3700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:02.748075 3700 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:34745
I20260809 23:14:02.748153 3925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:34745 every 8 connection(s)
I20260809 23:14:02.748430 3700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:14:02 dist-test-slave-7vjv krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317242, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:02.756153 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:02.749045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:40539 (local address 127.2.76.254:45837)
0809 23:14:02.749111 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:02.749112 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:02.749265 (+ 153us) server_negotiation.cc:407] Connection header received
0809 23:14:02.749963 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:02.749964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:02.749972 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:02.750019 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:02.750472 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.750887 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.751752 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.751916 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.751960 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:02.753763 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:02.753765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:02.753765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:02.753771 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:02.755093 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:02.755429 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:02.755429 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:02.755430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:02.755476 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:02.755664 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:02.755664 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:02.755664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:02.755751 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:02.755795 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:02.755988 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 23:14:02.756047 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 23:14:02.756336 3931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:02.749023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45837 (local address 127.2.76.194:40539)
0809 23:14:02.749160 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:02.749193 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:02.749887 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:02.750062 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:02.750065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:02.750260 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:02.750410 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:02.750413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.750956 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.750957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:02.751680 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:02.751682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.751981 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.751981 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:02.752071 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:02.752475 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:02.752482 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:02.753655 (+ 1173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:02.755163 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:02.755169 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:02.755171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:02.755372 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:02.755529 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:02.755530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:02.755530 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:02.755614 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:02.755799 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:02.755803 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:02.755893 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 23:14:02.755963 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 23:14:02.756065 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260809 23:14:02.756750 3926 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45837
I20260809 23:14:02.756855 3926 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:02.756910 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 3700
I20260809 23:14:02.757005 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:14:02.757030 3926 heartbeater.cc:511] Master 127.2.76.254:45837 requested a full tablet report, sending...
I20260809 23:14:02.757150 2355 external_mini_cluster.cc:1428] Setting key 40a3a1a960a5814eaad464da01b185e2
I20260809 23:14:02.757429 3349 ts_manager.cc:194] Registered new tserver with Master: 34bcded5bfa3400292fc15686f68e48c (127.2.76.194:34745)
I20260809 23:14:02.757920 3349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:40539
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:14:02.780383 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45837
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:02.856876 3934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:02.857071 3934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:02.857106 3934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:02.858151 3934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:02.858209 3934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:02.858248 3934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:14:02.859534 3934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45837
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:02.859795 3934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:02.860000 3934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:02.864463 3950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:02.864472 3946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:02.864480 3945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:02.864822 3934 server_base.cc:1034] running on GCE node
I20260809 23:14:02.865072 3934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:02.865281 3934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:02.866411 3934 hybrid_clock.cc:648] HybridClock initialized: now 1786317242866393 us; error 33 us; skew 500 ppm
Aug 09 23:14:02 dist-test-slave-7vjv krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317242, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:02.869422 3934 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:14:02.869871 3934 webserver.cc:460] Webserver started at http://127.2.76.195:37561/ using document root <none> and password file <none>
I20260809 23:14:02.870108 3934 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:02.870162 3934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:02.870288 3934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:02.871171 3934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "306a8cc0abf34e5b8dcbead6ae20d8f8"
format_stamp: "Formatted at 2026-08-09 23:14:02 on dist-test-slave-7vjv"
server_key: "aae5dc10e079f4434d66294a045687b3"
server_key_iv: "d303aebf6cfba81dce025c9b515e6637"
server_key_version: "encryptionkey@0"
I20260809 23:14:02.871518 3934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "306a8cc0abf34e5b8dcbead6ae20d8f8"
format_stamp: "Formatted at 2026-08-09 23:14:02 on dist-test-slave-7vjv"
server_key: "aae5dc10e079f4434d66294a045687b3"
server_key_iv: "d303aebf6cfba81dce025c9b515e6637"
server_key_version: "encryptionkey@0"
I20260809 23:14:02.872797 3934 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:14:02.873427 3959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:02.873606 3934 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:02.873672 3934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "306a8cc0abf34e5b8dcbead6ae20d8f8"
format_stamp: "Formatted at 2026-08-09 23:14:02 on dist-test-slave-7vjv"
server_key: "aae5dc10e079f4434d66294a045687b3"
server_key_iv: "d303aebf6cfba81dce025c9b515e6637"
server_key_version: "encryptionkey@0"
I20260809 23:14:02.873751 3934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:02.882292 3934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:02.884470 3934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:02.884629 3934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:02.884837 3934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:02.885116 3934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:02.885149 3934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:02.885169 3934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:02.885213 3934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:02.891623 3934 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:37111
I20260809 23:14:02.891782 4151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:37111 every 8 connection(s)
I20260809 23:14:02.891988 3934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:14:02.894505 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 3934
I20260809 23:14:02.894605 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:14:02.894773 2355 external_mini_cluster.cc:1428] Setting key 80cff63aca53de69674c03602e7cad99
Aug 09 23:14:02 dist-test-slave-7vjv krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317242, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:02.900727 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:02.892752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:34787 (local address 127.2.76.254:45837)
0809 23:14:02.892817 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:02.892819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:02.892986 (+ 167us) server_negotiation.cc:407] Connection header received
0809 23:14:02.893799 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:02.893800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:02.893810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:02.893860 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:02.894388 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.895076 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.896279 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.896448 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.896499 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:02.898174 (+ 1675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:02.898175 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:02.898176 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:02.898181 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:02.899610 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:02.899950 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:02.899950 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:02.899951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:02.899996 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:02.900244 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:02.900245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:02.900245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:02.900329 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:02.900376 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:02.900593 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 23:14:02.900649 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 23:14:02.900981 4154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:02.892719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45837 (local address 127.2.76.195:34787)
0809 23:14:02.892862 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:02.892897 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:02.893716 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:02.893905 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:02.893908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:02.894126 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:02.894286 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:02.894289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.895153 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.895154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:02.896212 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:02.896214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.896517 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.896517 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:02.896601 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:02.896945 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:02.896952 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:02.898076 (+ 1124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:02.899679 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:02.899684 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:02.899685 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:02.899891 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:02.900049 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:02.900050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:02.900050 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:02.900196 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:02.900380 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:02.900384 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:02.900476 (+ 92us) client_negotiation.cc:769] Sending connection context
0809 23:14:02.900569 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 23:14:02.900673 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260809 23:14:02.901473 4152 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45837
I20260809 23:14:02.901556 4152 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:02.901737 4152 heartbeater.cc:511] Master 127.2.76.254:45837 requested a full tablet report, sending...
I20260809 23:14:02.902153 3349 ts_manager.cc:194] Registered new tserver with Master: 306a8cc0abf34e5b8dcbead6ae20d8f8 (127.2.76.195:37111)
I20260809 23:14:02.902441 3349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:34787
I20260809 23:14:02.906363 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:14:02.913084 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:02.908131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55470 (local address 127.2.76.254:45837)
0809 23:14:02.908173 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:02.908175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:02.908254 (+ 79us) server_negotiation.cc:407] Connection header received
0809 23:14:02.908287 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:02.908287 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:02.908294 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:02.908332 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:02.908794 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.909174 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.909791 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.909930 (+ 139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.909968 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:02.910924 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:02.910925 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:02.910925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:02.910930 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:02.912234 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:02.912518 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:02.912518 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:02.912519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:02.912583 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:02.912698 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:02.912698 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:02.912698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:02.912805 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:02.912869 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:02.912928 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 23:14:02.912980 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:14:02.915227 3349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55470:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:14:02.917042 3349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:14:02.923448 4166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:02.919631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:34745 (local address 127.0.0.1:43204)
0809 23:14:02.919865 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:02.919874 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:02.919913 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:02.920310 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:02.920311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:02.920314 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:02.920384 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:02.920386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.921618 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.921619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:02.922285 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:02.922287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.923135 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.923136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:02.923232 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:02.923247 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 23:14:02.923328 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 23:14:02.923381 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":68,"threads_started":1}
I20260809 23:14:02.923529 4168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:02.919683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39248 (local address 127.2.76.195:37111)
0809 23:14:02.920303 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:02.920306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:02.920314 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:02.920343 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:02.920344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:02.920357 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:02.920469 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:02.920568 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.921459 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.922608 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.923178 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.923259 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:02.923330 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:02.923402 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 23:14:02.923437 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":63,"threads_started":1}
I20260809 23:14:02.923718 4172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:02.919924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43986 (local address 127.2.76.193:35983)
0809 23:14:02.920108 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:02.920113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:02.920124 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:14:02.920163 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:02.920165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:02.920181 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:02.920290 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:02.920425 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.921197 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.922478 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.923155 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.923496 (+ 341us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:02.923565 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:02.923594 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 23:14:02.923639 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
I20260809 23:14:02.923580 4164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:02.919620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:35983 (local address 127.0.0.1:43986)
0809 23:14:02.919792 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:02.919825 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:02.919887 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:02.920301 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:02.920305 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:02.920307 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:02.920365 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:02.920367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.921300 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.921301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:02.922384 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:02.922387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.923300 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.923301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:02.923367 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:02.923372 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:14:02.923406 (+ 34us) client_negotiation.cc:240] Negotiation successful
0809 23:14:02.923449 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":66,"threads_started":1}
I20260809 23:14:02.923655 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:02.919620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:37111 (local address 127.0.0.1:39248)
0809 23:14:02.919983 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:02.919992 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:02.920039 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:02.920471 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:02.920472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:02.920473 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:02.920511 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:02.920512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.921558 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.921559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:02.922523 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:02.922525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.923256 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.923257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:02.923338 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:02.923341 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:14:02.923391 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 23:14:02.923430 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":20,"spinlock_wait_cycles":384,"thread_start_us":75,"threads_started":1}
I20260809 23:14:02.923991 4169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:02.919872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43204 (local address 127.2.76.194:34745)
0809 23:14:02.920061 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:02.920065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:02.920075 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:14:02.920129 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:02.920132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:02.920146 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:02.920215 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:02.920470 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.921482 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.922370 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:02.923040 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:02.923130 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:02.923187 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:02.923844 (+ 657us) server_negotiation.cc:299] Negotiation successful
0809 23:14:02.923895 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":106,"threads_started":1}
I20260809 23:14:02.924625 4037 tablet_service.cc:1467] Processing CreateTablet for tablet b5f68422331e41b6bf048050fbc88138 (DEFAULT_TABLE table=test-table [id=eaedc566a26743c083593a67d9b9ad70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:02.924625 3568 tablet_service.cc:1467] Processing CreateTablet for tablet b5f68422331e41b6bf048050fbc88138 (DEFAULT_TABLE table=test-table [id=eaedc566a26743c083593a67d9b9ad70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:02.924816 3796 tablet_service.cc:1467] Processing CreateTablet for tablet b5f68422331e41b6bf048050fbc88138 (DEFAULT_TABLE table=test-table [id=eaedc566a26743c083593a67d9b9ad70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:02.924944 3568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5f68422331e41b6bf048050fbc88138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:02.924944 4037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5f68422331e41b6bf048050fbc88138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:02.925115 3796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5f68422331e41b6bf048050fbc88138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:02.927284 4175 tablet_bootstrap.cc:492] T b5f68422331e41b6bf048050fbc88138 P 34bcded5bfa3400292fc15686f68e48c: Bootstrap starting.
I20260809 23:14:02.928128 4177 tablet_bootstrap.cc:492] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375: Bootstrap starting.
I20260809 23:14:02.928417 4175 tablet_bootstrap.cc:654] T b5f68422331e41b6bf048050fbc88138 P 34bcded5bfa3400292fc15686f68e48c: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:02.928778 4177 tablet_bootstrap.cc:654] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:02.929104 4175 log.cc:824] T b5f68422331e41b6bf048050fbc88138 P 34bcded5bfa3400292fc15686f68e48c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:02.929420 4177 log.cc:824] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:02.929678 4176 tablet_bootstrap.cc:492] T b5f68422331e41b6bf048050fbc88138 P 306a8cc0abf34e5b8dcbead6ae20d8f8: Bootstrap starting.
I20260809 23:14:02.929837 4175 tablet_bootstrap.cc:492] T b5f68422331e41b6bf048050fbc88138 P 34bcded5bfa3400292fc15686f68e48c: No bootstrap required, opened a new log
I20260809 23:14:02.929908 4175 ts_tablet_manager.cc:1397] T b5f68422331e41b6bf048050fbc88138 P 34bcded5bfa3400292fc15686f68e48c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 23:14:02.930100 4177 tablet_bootstrap.cc:492] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375: No bootstrap required, opened a new log
I20260809 23:14:02.930162 4177 ts_tablet_manager.cc:1397] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:02.930372 4176 tablet_bootstrap.cc:654] T b5f68422331e41b6bf048050fbc88138 P 306a8cc0abf34e5b8dcbead6ae20d8f8: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:02.930790 4176 log.cc:824] T b5f68422331e41b6bf048050fbc88138 P 306a8cc0abf34e5b8dcbead6ae20d8f8: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:02.931450 4176 tablet_bootstrap.cc:492] T b5f68422331e41b6bf048050fbc88138 P 306a8cc0abf34e5b8dcbead6ae20d8f8: No bootstrap required, opened a new log
I20260809 23:14:02.931532 4176 ts_tablet_manager.cc:1397] T b5f68422331e41b6bf048050fbc88138 P 306a8cc0abf34e5b8dcbead6ae20d8f8: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 23:14:02.931859 4175 raft_consensus.cc:348] T b5f68422331e41b6bf048050fbc88138 P 34bcded5bfa3400292fc15686f68e48c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bcded5bfa3400292fc15686f68e48c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34745 } } peers { permanent_uuid: "306a8cc0abf34e5b8dcbead6ae20d8f8" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37111 } } peers { permanent_uuid: "216698123d00495a84e51def125fe375" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35983 } }
I20260809 23:14:02.932039 4175 raft_consensus.cc:374] T b5f68422331e41b6bf048050fbc88138 P 34bcded5bfa3400292fc15686f68e48c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:02.932072 4175 raft_consensus.cc:729] T b5f68422331e41b6bf048050fbc88138 P 34bcded5bfa3400292fc15686f68e48c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34bcded5bfa3400292fc15686f68e48c, State: Initialized, Role: FOLLOWER
I20260809 23:14:02.932216 4175 consensus_queue.cc:260] T b5f68422331e41b6bf048050fbc88138 P 34bcded5bfa3400292fc15686f68e48c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bcded5bfa3400292fc15686f68e48c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34745 } } peers { permanent_uuid: "306a8cc0abf34e5b8dcbead6ae20d8f8" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37111 } } peers { permanent_uuid: "216698123d00495a84e51def125fe375" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35983 } }
I20260809 23:14:02.932219 4177 raft_consensus.cc:348] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bcded5bfa3400292fc15686f68e48c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34745 } } peers { permanent_uuid: "306a8cc0abf34e5b8dcbead6ae20d8f8" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37111 } } peers { permanent_uuid: "216698123d00495a84e51def125fe375" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35983 } }
I20260809 23:14:02.932374 4177 raft_consensus.cc:374] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:02.932411 4177 raft_consensus.cc:729] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 216698123d00495a84e51def125fe375, State: Initialized, Role: FOLLOWER
I20260809 23:14:02.932513 4175 ts_tablet_manager.cc:1428] T b5f68422331e41b6bf048050fbc88138 P 34bcded5bfa3400292fc15686f68e48c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:14:02.932546 4177 consensus_queue.cc:260] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bcded5bfa3400292fc15686f68e48c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34745 } } peers { permanent_uuid: "306a8cc0abf34e5b8dcbead6ae20d8f8" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37111 } } peers { permanent_uuid: "216698123d00495a84e51def125fe375" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35983 } }
I20260809 23:14:02.932665 3926 heartbeater.cc:503] Master 127.2.76.254:45837 was elected leader, sending a full tablet report...
I20260809 23:14:02.932838 4177 ts_tablet_manager.cc:1428] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 23:14:02.932952 3686 heartbeater.cc:503] Master 127.2.76.254:45837 was elected leader, sending a full tablet report...
I20260809 23:14:02.933486 4176 raft_consensus.cc:348] T b5f68422331e41b6bf048050fbc88138 P 306a8cc0abf34e5b8dcbead6ae20d8f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bcded5bfa3400292fc15686f68e48c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34745 } } peers { permanent_uuid: "306a8cc0abf34e5b8dcbead6ae20d8f8" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37111 } } peers { permanent_uuid: "216698123d00495a84e51def125fe375" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35983 } }
I20260809 23:14:02.933640 4176 raft_consensus.cc:374] T b5f68422331e41b6bf048050fbc88138 P 306a8cc0abf34e5b8dcbead6ae20d8f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:02.933681 4176 raft_consensus.cc:729] T b5f68422331e41b6bf048050fbc88138 P 306a8cc0abf34e5b8dcbead6ae20d8f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 306a8cc0abf34e5b8dcbead6ae20d8f8, State: Initialized, Role: FOLLOWER
I20260809 23:14:02.933799 4176 consensus_queue.cc:260] T b5f68422331e41b6bf048050fbc88138 P 306a8cc0abf34e5b8dcbead6ae20d8f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bcded5bfa3400292fc15686f68e48c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34745 } } peers { permanent_uuid: "306a8cc0abf34e5b8dcbead6ae20d8f8" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37111 } } peers { permanent_uuid: "216698123d00495a84e51def125fe375" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35983 } }
I20260809 23:14:02.934067 4176 ts_tablet_manager.cc:1428] T b5f68422331e41b6bf048050fbc88138 P 306a8cc0abf34e5b8dcbead6ae20d8f8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:14:02.934099 4152 heartbeater.cc:503] Master 127.2.76.254:45837 was elected leader, sending a full tablet report...
W20260809 23:14:02.998903 3929 tablet.cc:2369] T b5f68422331e41b6bf048050fbc88138 P 34bcded5bfa3400292fc15686f68e48c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:14:03.114439 4188 raft_consensus.cc:482] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:14:03.114563 4188 raft_consensus.cc:504] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bcded5bfa3400292fc15686f68e48c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34745 } } peers { permanent_uuid: "306a8cc0abf34e5b8dcbead6ae20d8f8" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37111 } } peers { permanent_uuid: "216698123d00495a84e51def125fe375" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35983 } }
I20260809 23:14:03.115059 4188 leader_election.cc:288] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34bcded5bfa3400292fc15686f68e48c (127.2.76.194:34745), 306a8cc0abf34e5b8dcbead6ae20d8f8 (127.2.76.195:37111)
W20260809 23:14:03.115440 3689 tablet.cc:2369] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:14:03.118079 3693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:03.115225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:37111 (local address 127.2.76.193:43239)
0809 23:14:03.115278 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:03.115290 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:03.115328 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:03.115497 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:03.115498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:03.115499 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:03.115567 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:03.115569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:03.116362 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:03.116363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:03.117239 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:03.117242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:03.117830 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:03.117832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:03.117920 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:03.117923 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:14:03.117962 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 23:14:03.117988 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":76}
I20260809 23:14:03.118134 4168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:03.115333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:43239 (local address 127.2.76.195:37111)
0809 23:14:03.115389 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:03.115391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:03.115399 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:03.115425 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:03.115426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:03.115436 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:03.115484 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:03.115637 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:03.116286 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:03.117369 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:03.117723 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:03.117831 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:03.117840 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:03.117990 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 23:14:03.118045 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:14:03.118180 4169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:03.115323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:40629 (local address 127.2.76.194:34745)
0809 23:14:03.115407 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:03.115409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:03.115418 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:14:03.115484 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:03.115484 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:03.115495 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:03.115539 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:03.115821 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:03.116359 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:03.117328 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:03.117724 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:03.117833 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:03.117841 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:03.118070 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 23:14:03.118111 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 23:14:03.118216 4190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:03.115237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:34745 (local address 127.2.76.193:40629)
0809 23:14:03.115355 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:03.115366 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:03.115414 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:03.115554 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:03.115555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:03.115558 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:03.115740 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:03.115742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:03.116431 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:03.116432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:03.117237 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:03.117239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:03.117832 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:03.117833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:03.117989 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:03.117993 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:14:03.118049 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 23:14:03.118082 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":1,"thread_start_us":60,"threads_started":1}
I20260809 23:14:03.118563 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5f68422331e41b6bf048050fbc88138" candidate_uuid: "216698123d00495a84e51def125fe375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "306a8cc0abf34e5b8dcbead6ae20d8f8" is_pre_election: true
I20260809 23:14:03.118600 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5f68422331e41b6bf048050fbc88138" candidate_uuid: "216698123d00495a84e51def125fe375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34bcded5bfa3400292fc15686f68e48c" is_pre_election: true
I20260809 23:14:03.118752 3848 raft_consensus.cc:2454] T b5f68422331e41b6bf048050fbc88138 P 34bcded5bfa3400292fc15686f68e48c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 216698123d00495a84e51def125fe375 in term 0.
I20260809 23:14:03.118763 4069 raft_consensus.cc:2454] T b5f68422331e41b6bf048050fbc88138 P 306a8cc0abf34e5b8dcbead6ae20d8f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 216698123d00495a84e51def125fe375 in term 0.
I20260809 23:14:03.119055 3494 leader_election.cc:302] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 216698123d00495a84e51def125fe375, 34bcded5bfa3400292fc15686f68e48c; no voters:
I20260809 23:14:03.119182 4188 raft_consensus.cc:2781] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:14:03.119251 4188 raft_consensus.cc:482] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:14:03.119274 4188 raft_consensus.cc:3037] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:03.119951 4188 raft_consensus.cc:504] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bcded5bfa3400292fc15686f68e48c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34745 } } peers { permanent_uuid: "306a8cc0abf34e5b8dcbead6ae20d8f8" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37111 } } peers { permanent_uuid: "216698123d00495a84e51def125fe375" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35983 } }
I20260809 23:14:03.120112 4188 leader_election.cc:288] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [CANDIDATE]: Term 1 election: Requested vote from peers 34bcded5bfa3400292fc15686f68e48c (127.2.76.194:34745), 306a8cc0abf34e5b8dcbead6ae20d8f8 (127.2.76.195:37111)
I20260809 23:14:03.120239 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5f68422331e41b6bf048050fbc88138" candidate_uuid: "216698123d00495a84e51def125fe375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34bcded5bfa3400292fc15686f68e48c"
I20260809 23:14:03.120344 3848 raft_consensus.cc:3037] T b5f68422331e41b6bf048050fbc88138 P 34bcded5bfa3400292fc15686f68e48c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:03.121101 4069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5f68422331e41b6bf048050fbc88138" candidate_uuid: "216698123d00495a84e51def125fe375" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "306a8cc0abf34e5b8dcbead6ae20d8f8"
I20260809 23:14:03.121166 4069 raft_consensus.cc:3037] T b5f68422331e41b6bf048050fbc88138 P 306a8cc0abf34e5b8dcbead6ae20d8f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:03.121428 3848 raft_consensus.cc:2454] T b5f68422331e41b6bf048050fbc88138 P 34bcded5bfa3400292fc15686f68e48c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 216698123d00495a84e51def125fe375 in term 1.
I20260809 23:14:03.121666 3494 leader_election.cc:302] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 216698123d00495a84e51def125fe375, 34bcded5bfa3400292fc15686f68e48c; no voters:
I20260809 23:14:03.121804 4188 raft_consensus.cc:2781] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:03.121984 4188 raft_consensus.cc:686] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [term 1 LEADER]: Becoming Leader. State: Replica: 216698123d00495a84e51def125fe375, State: Running, Role: LEADER
I20260809 23:14:03.122099 4069 raft_consensus.cc:2454] T b5f68422331e41b6bf048050fbc88138 P 306a8cc0abf34e5b8dcbead6ae20d8f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 216698123d00495a84e51def125fe375 in term 1.
I20260809 23:14:03.122093 4188 consensus_queue.cc:237] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bcded5bfa3400292fc15686f68e48c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34745 } } peers { permanent_uuid: "306a8cc0abf34e5b8dcbead6ae20d8f8" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37111 } } peers { permanent_uuid: "216698123d00495a84e51def125fe375" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35983 } }
I20260809 23:14:03.123062 3348 catalog_manager.cc:5495] T b5f68422331e41b6bf048050fbc88138 P 216698123d00495a84e51def125fe375 reported cstate change: term changed from 0 to 1, leader changed from <none> to 216698123d00495a84e51def125fe375 (127.2.76.193). New cstate: current_term: 1 leader_uuid: "216698123d00495a84e51def125fe375" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34bcded5bfa3400292fc15686f68e48c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "306a8cc0abf34e5b8dcbead6ae20d8f8" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "216698123d00495a84e51def125fe375" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35983 } health_report { overall_health: HEALTHY } } }
W20260809 23:14:03.142661 4153 tablet.cc:2369] T b5f68422331e41b6bf048050fbc88138 P 306a8cc0abf34e5b8dcbead6ae20d8f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 23:14:03 dist-test-slave-7vjv krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.193@KRBTEST.COM
I20260809 23:14:03.150529 4172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:03.143832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44000 (local address 127.2.76.193:35983)
0809 23:14:03.143884 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:03.143886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:03.144085 (+ 199us) server_negotiation.cc:407] Connection header received
0809 23:14:03.144170 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:03.144171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:03.144181 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:03.144229 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:03.144794 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:03.145171 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:03.145776 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:03.145919 (+ 143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:03.145959 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:03.147934 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:03.147941 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:03.147942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:03.147953 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:03.149472 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:03.149808 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:03.149811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:03.149811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:03.149878 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:03.150066 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:03.150067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:03.150067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:03.150184 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:03.150307 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:03.150376 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 23:14:03.150446 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Aug 09 23:14:03 dist-test-slave-7vjv krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317243, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 23:14:03 dist-test-slave-7vjv krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317243, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.76.193@KRBTEST.COM
I20260809 23:14:03.173388 4172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:03.166991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44012 (local address 127.2.76.193:35983)
0809 23:14:03.167040 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:03.167041 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:03.167084 (+ 43us) server_negotiation.cc:407] Connection header received
0809 23:14:03.167116 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:03.167117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:03.167123 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:03.167161 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:03.167626 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:03.168012 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:03.168665 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:03.168826 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:03.168872 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:03.170805 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:03.170806 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:03.170806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:03.170814 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:03.172268 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:03.172604 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:03.172605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:03.172606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:03.172660 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:03.172849 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:03.172850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:03.172850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:03.172939 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:03.173054 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:03.173236 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 23:14:03.173301 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Aug 09 23:14:03 dist-test-slave-7vjv krb5kdc[3282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 23:14:03 dist-test-slave-7vjv krb5kdc[3282](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.193@KRBTEST.COM
I20260809 23:14:03.194799 4172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:03.188882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44020 (local address 127.2.76.193:35983)
0809 23:14:03.188933 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:03.188935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:03.188988 (+ 53us) server_negotiation.cc:407] Connection header received
0809 23:14:03.189023 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:03.189024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:03.189031 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:03.189072 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:03.189513 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:03.189873 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:03.190462 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:03.190617 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:03.190659 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:03.192432 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:03.192433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:03.192433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:03.192438 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:03.193746 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:03.194062 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:03.194063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:03.194063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:03.194138 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:03.194336 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:03.194336 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:03.194337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:03.194430 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:03.194528 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:03.194642 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 23:14:03.194710 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 23:14:03.195407 3532 tablet_service.cc:2908] Rejecting scan request for tablet b5f68422331e41b6bf048050fbc88138: Uninitialized: clean time has not yet been initialized
I20260809 23:14:03.196285 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 3466
I20260809 23:14:03.211452 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 3700
I20260809 23:14:03.222472 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 3934
I20260809 23:14:03.231848 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 3303
2026-08-09T23:14:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2200 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:14:03 dist-test-slave-7vjv krb5kdc[4216](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:14:03 dist-test-slave-7vjv krb5kdc[4216](info): set up 2 sockets
Aug 09 23:14:03 dist-test-slave-7vjv krb5kdc[4216](info): commencing operation
krb5kdc: starting...
W20260809 23:14:04.375133 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:14:04 dist-test-slave-7vjv krb5kdc[4216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:14:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:14:04.480950 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:39573
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:36919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:39573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:04.554003 4235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:04.554199 4235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:04.554236 4235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:04.555328 4235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:04.555365 4235 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 23:14:04.555377 4235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:04.555388 4235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:04.555431 4235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:04.555446 4235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:14:04.556778 4235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:36919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:39573
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:39573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:04.556998 4235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:04.557193 4235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:04.561200 4243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:04.561321 4235 server_base.cc:1034] running on GCE node
W20260809 23:14:04.561228 4244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:04.561254 4248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:04.561699 4235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:04.561973 4235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:04.563118 4235 hybrid_clock.cc:648] HybridClock initialized: now 1786317244563111 us; error 30 us; skew 500 ppm
Aug 09 23:14:04 dist-test-slave-7vjv krb5kdc[4216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317244, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:04.565238 4235 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:14:04.565702 4235 webserver.cc:460] Webserver started at http://127.2.76.254:36379/ using document root <none> and password file <none>
I20260809 23:14:04.565908 4235 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:04.565971 4235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:04.566110 4235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:04.566942 4235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "da146264d5354636a20a4c850e8c709d"
format_stamp: "Formatted at 2026-08-09 23:14:04 on dist-test-slave-7vjv"
server_key: "f054ab629bbc510fcb7bbd779f9def0f"
server_key_iv: "94d6a8165722bbdddf7cf821ccedfda3"
server_key_version: "encryptionkey@0"
I20260809 23:14:04.567260 4235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "da146264d5354636a20a4c850e8c709d"
format_stamp: "Formatted at 2026-08-09 23:14:04 on dist-test-slave-7vjv"
server_key: "f054ab629bbc510fcb7bbd779f9def0f"
server_key_iv: "94d6a8165722bbdddf7cf821ccedfda3"
server_key_version: "encryptionkey@0"
I20260809 23:14:04.568552 4235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 23:14:04.569203 4265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:04.569352 4235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:04.569440 4235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "da146264d5354636a20a4c850e8c709d"
format_stamp: "Formatted at 2026-08-09 23:14:04 on dist-test-slave-7vjv"
server_key: "f054ab629bbc510fcb7bbd779f9def0f"
server_key_iv: "94d6a8165722bbdddf7cf821ccedfda3"
server_key_version: "encryptionkey@0"
I20260809 23:14:04.569509 4235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:04.586042 4235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:04.587630 4235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:04.587786 4235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:04.592587 4235 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:39573
I20260809 23:14:04.592630 4370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:39573 every 8 connection(s)
I20260809 23:14:04.592952 4235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:04.593477 4371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:04.594872 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 4235
I20260809 23:14:04.595002 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:04.595157 2355 external_mini_cluster.cc:1428] Setting key da7e8148b1967b25e151975db5b7c525
I20260809 23:14:04.595820 4371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d: Bootstrap starting.
I20260809 23:14:04.596390 4371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:04.596675 4371 log.cc:824] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:04.597296 4371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d: No bootstrap required, opened a new log
I20260809 23:14:04.598507 4371 raft_consensus.cc:348] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da146264d5354636a20a4c850e8c709d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39573 } }
I20260809 23:14:04.598654 4371 raft_consensus.cc:374] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:04.598714 4371 raft_consensus.cc:729] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da146264d5354636a20a4c850e8c709d, State: Initialized, Role: FOLLOWER
I20260809 23:14:04.598847 4371 consensus_queue.cc:260] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da146264d5354636a20a4c850e8c709d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39573 } }
I20260809 23:14:04.598932 4371 raft_consensus.cc:388] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:04.598974 4371 raft_consensus.cc:482] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:04.599026 4371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:04.599462 4371 raft_consensus.cc:504] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da146264d5354636a20a4c850e8c709d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39573 } }
I20260809 23:14:04.599597 4371 leader_election.cc:302] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da146264d5354636a20a4c850e8c709d; no voters:
I20260809 23:14:04.599779 4371 leader_election.cc:288] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:04.599819 4377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:04.599984 4377 raft_consensus.cc:686] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [term 1 LEADER]: Becoming Leader. State: Replica: da146264d5354636a20a4c850e8c709d, State: Running, Role: LEADER
I20260809 23:14:04.600019 4371 sys_catalog.cc:564] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:04.600167 4377 consensus_queue.cc:237] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da146264d5354636a20a4c850e8c709d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39573 } }
I20260809 23:14:04.601497 4379 sys_catalog.cc:455] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [sys.catalog]: SysCatalogTable state changed. Reason: New leader da146264d5354636a20a4c850e8c709d. Latest consensus state: current_term: 1 leader_uuid: "da146264d5354636a20a4c850e8c709d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da146264d5354636a20a4c850e8c709d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39573 } } }
W20260809 23:14:04.601766 4402 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 23:14:04.601886 4379 sys_catalog.cc:458] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:04.600737 4378 sys_catalog.cc:455] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da146264d5354636a20a4c850e8c709d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da146264d5354636a20a4c850e8c709d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39573 } } }
I20260809 23:14:04.602087 4378 sys_catalog.cc:458] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d [sys.catalog]: This master's current role is: LEADER
W20260809 23:14:04.602159 4402 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:14:04.602284 4405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:04.602451 4405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 23:14:04 dist-test-slave-7vjv krb5kdc[4216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:04.603866 4405 catalog_manager.cc:1347] Generated new cluster ID: 97c8257369784e7ea3a540310b4a7d84
I20260809 23:14:04.603911 4405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:14:04.606046 4375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:04.595616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51984 (local address 127.2.76.254:39573)
0809 23:14:04.595794 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:04.595808 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:04.595847 (+ 39us) server_negotiation.cc:407] Connection header received
0809 23:14:04.596336 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:04.596342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:04.596384 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:04.596526 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:04.597156 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.598744 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.599532 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.600175 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.600274 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:04.603048 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:04.603052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:04.603052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:04.603062 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:04.604651 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:04.604999 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:04.605002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:04.605002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:04.605079 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:04.605267 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:04.605268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:04.605268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:04.605466 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:04.605631 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:04.605731 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 23:14:04.605856 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":75,"threads_started":1}
I20260809 23:14:04.611126 4405 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:04.611562 4405 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:04.616518 4405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d: Generated new TSK 0
I20260809 23:14:04.616662 4405 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:14:04.648285 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:39573
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:36919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:04.731982 4415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:04.732232 4415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:04.732259 4415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:04.733389 4415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:04.733435 4415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:04.733455 4415 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:14:04.733472 4415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:14:04.734782 4415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:36919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.76.254:39573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:04.734999 4415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:04.735262 4415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:04.740339 4430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:04.740339 4431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:04.740475 4436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:04.740669 4415 server_base.cc:1034] running on GCE node
I20260809 23:14:04.740885 4415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:04.741138 4415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:04.742288 4415 hybrid_clock.cc:648] HybridClock initialized: now 1786317244742275 us; error 35 us; skew 500 ppm
Aug 09 23:14:04 dist-test-slave-7vjv krb5kdc[4216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317244, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:04.744743 4415 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:14:04.745209 4415 webserver.cc:460] Webserver started at http://127.2.76.193:39919/ using document root <none> and password file <none>
I20260809 23:14:04.745383 4415 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:04.745419 4415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:04.745514 4415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:04.746363 4415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "c201a739bce54c23b7add3aa5e9847f9"
format_stamp: "Formatted at 2026-08-09 23:14:04 on dist-test-slave-7vjv"
server_key: "c7cf6ad3043d20d80645c1e5090a329a"
server_key_iv: "eea837790418d3a09404c4f1a27588cd"
server_key_version: "encryptionkey@0"
I20260809 23:14:04.746634 4415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "c201a739bce54c23b7add3aa5e9847f9"
format_stamp: "Formatted at 2026-08-09 23:14:04 on dist-test-slave-7vjv"
server_key: "c7cf6ad3043d20d80645c1e5090a329a"
server_key_iv: "eea837790418d3a09404c4f1a27588cd"
server_key_version: "encryptionkey@0"
I20260809 23:14:04.747781 4415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:14:04.748445 4446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:04.748605 4415 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:04.748696 4415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "c201a739bce54c23b7add3aa5e9847f9"
format_stamp: "Formatted at 2026-08-09 23:14:04 on dist-test-slave-7vjv"
server_key: "c7cf6ad3043d20d80645c1e5090a329a"
server_key_iv: "eea837790418d3a09404c4f1a27588cd"
server_key_version: "encryptionkey@0"
I20260809 23:14:04.748765 4415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:04.758090 4415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:04.760187 4415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:04.760317 4415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:04.760769 4415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:04.760810 4415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:04.760830 4415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:04.760842 4415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:14:04 dist-test-slave-7vjv krb5kdc[4216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317244, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:04.767486 4415 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:36215
I20260809 23:14:04.767791 4415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:14:04.767807 4661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:36215 every 8 connection(s)
I20260809 23:14:04.769912 4375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:04.761197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:33157 (local address 127.2.76.254:39573)
0809 23:14:04.761265 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:04.761268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:04.761462 (+ 194us) server_negotiation.cc:407] Connection header received
0809 23:14:04.761902 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:04.761903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:04.761917 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:04.761960 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:04.762722 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.763503 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.764667 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.764865 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.764948 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:04.767250 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:04.767252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:04.767252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:04.767264 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:04.768700 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:04.769058 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:04.769059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:04.769059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:04.769115 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:04.769336 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:04.769337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:04.769338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:04.769444 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:04.769512 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:04.769755 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 23:14:04.769834 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 23:14:04.770036 4461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:04.761176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:39573 (local address 127.2.76.193:33157)
0809 23:14:04.761350 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:04.761382 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:04.761765 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:04.762043 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:04.762046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:04.762362 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:04.762612 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:04.762619 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.763615 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.763616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:04.764577 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:04.764580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.764972 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.764973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:04.765084 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:04.765524 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:04.765531 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:04.767154 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:04.768784 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:04.768790 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:04.768792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:04.768980 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:04.769178 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:04.769179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:04.769179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:04.769277 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:04.769513 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:04.769520 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:04.769635 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 23:14:04.769724 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 23:14:04.769854 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":106,"threads_started":1}
I20260809 23:14:04.770445 4662 heartbeater.cc:347] Connected to a master server at 127.2.76.254:39573
I20260809 23:14:04.770525 4662 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:04.770673 4662 heartbeater.cc:511] Master 127.2.76.254:39573 requested a full tablet report, sending...
I20260809 23:14:04.771150 4293 ts_manager.cc:194] Registered new tserver with Master: c201a739bce54c23b7add3aa5e9847f9 (127.2.76.193:36215)
I20260809 23:14:04.771989 4293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:33157
I20260809 23:14:04.772634 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 4415
I20260809 23:14:04.772753 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:14:04.772926 2355 external_mini_cluster.cc:1428] Setting key ede540f92e170af22c6febcf232018b0
I20260809 23:14:04.776837 4676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:04.771359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:39573 (local address 127.2.76.193:54925)
0809 23:14:04.771489 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:04.771497 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:04.771549 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:04.771697 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:04.771697 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:04.771870 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:04.771928 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:04.771929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.772905 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.772906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:04.773545 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:04.773548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.773921 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.773922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:04.774002 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:04.774387 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:04.774391 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:04.774618 (+ 227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:04.775925 (+ 1307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:04.775926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:04.775927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:04.776080 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:04.776298 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:04.776299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:04.776299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:04.776355 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:04.776618 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:04.776619 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:04.776657 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 23:14:04.776714 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 23:14:04.776783 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":60,"threads_started":1}
I20260809 23:14:04.776986 4375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:04.771400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:54925 (local address 127.2.76.254:39573)
0809 23:14:04.771435 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:04.771436 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:04.771539 (+ 103us) server_negotiation.cc:407] Connection header received
0809 23:14:04.771597 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:04.771598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:04.771605 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:04.771691 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:04.771992 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.772760 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.773657 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.773841 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.773932 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:04.774705 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:04.774706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:04.774706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:04.774712 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:04.775837 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:04.776176 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:04.776177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:04.776177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:04.776235 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:04.776423 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:04.776424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:04.776424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:04.776545 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:04.776606 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:04.776760 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 23:14:04.776856 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:14:04.796969 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:39573
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:36919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:04.869179 4683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:04.869369 4683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:04.869405 4683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:04.870488 4683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:04.870544 4683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:04.870576 4683 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:14:04.870625 4683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:14:04.871964 4683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:36919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.76.254:39573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:04.872219 4683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:04.872457 4683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:04.876784 4696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:04.876809 4692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:04.876784 4694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:04.877056 4683 server_base.cc:1034] running on GCE node
I20260809 23:14:04.877247 4683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:04.877485 4683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:04.878644 4683 hybrid_clock.cc:648] HybridClock initialized: now 1786317244878646 us; error 42 us; skew 500 ppm
Aug 09 23:14:04 dist-test-slave-7vjv krb5kdc[4216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317244, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:04.880813 4683 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:14:04.881250 4683 webserver.cc:460] Webserver started at http://127.2.76.194:34561/ using document root <none> and password file <none>
I20260809 23:14:04.881404 4683 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:04.881439 4683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:04.881516 4683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:04.882325 4683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "cd23852b8fb146bea7eba9caac34cdd5"
format_stamp: "Formatted at 2026-08-09 23:14:04 on dist-test-slave-7vjv"
server_key: "202d1ccd384cc49afbe1e4e662d9bd94"
server_key_iv: "1db031fe9062997764c9631be808f6a8"
server_key_version: "encryptionkey@0"
I20260809 23:14:04.882587 4683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "cd23852b8fb146bea7eba9caac34cdd5"
format_stamp: "Formatted at 2026-08-09 23:14:04 on dist-test-slave-7vjv"
server_key: "202d1ccd384cc49afbe1e4e662d9bd94"
server_key_iv: "1db031fe9062997764c9631be808f6a8"
server_key_version: "encryptionkey@0"
I20260809 23:14:04.883725 4683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:14:04.884357 4708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:04.884529 4683 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:14:04.884611 4683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "cd23852b8fb146bea7eba9caac34cdd5"
format_stamp: "Formatted at 2026-08-09 23:14:04 on dist-test-slave-7vjv"
server_key: "202d1ccd384cc49afbe1e4e662d9bd94"
server_key_iv: "1db031fe9062997764c9631be808f6a8"
server_key_version: "encryptionkey@0"
I20260809 23:14:04.884660 4683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:04.894601 4683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:04.896597 4683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:04.896695 4683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:04.897060 4683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:04.897097 4683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:04.897117 4683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:04.897169 4683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:14:04 dist-test-slave-7vjv krb5kdc[4216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317244, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:04.903899 4683 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:44373
I20260809 23:14:04.904265 4922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:44373 every 8 connection(s)
I20260809 23:14:04.904304 4683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:14:04.906006 4375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:04.897472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:38463 (local address 127.2.76.254:39573)
0809 23:14:04.897548 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:04.897550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:04.897732 (+ 182us) server_negotiation.cc:407] Connection header received
0809 23:14:04.898091 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:04.898092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:04.898107 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:04.898182 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:04.898850 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.899623 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.901042 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.901366 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.901444 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:04.903728 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:04.903729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:04.903730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:04.903737 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:04.905106 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:04.905435 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:04.905436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:04.905436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:04.905485 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:04.905625 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:04.905625 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:04.905625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:04.905703 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:04.905742 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:04.905898 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 23:14:04.905950 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 23:14:04.906185 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:04.897445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:39573 (local address 127.2.76.194:38463)
0809 23:14:04.897617 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:04.897651 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:04.898003 (+ 352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:04.898222 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:04.898224 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:04.898498 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:04.898752 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:04.898758 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.899723 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.899724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:04.900958 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:04.900960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.901465 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.901466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:04.901625 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:04.902166 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:04.902173 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:04.903611 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:04.905167 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:04.905171 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:04.905172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:04.905383 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:04.905523 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:04.905523 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:04.905524 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:04.905594 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:04.905743 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:04.905745 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:04.905828 (+ 83us) client_negotiation.cc:769] Sending connection context
0809 23:14:04.905889 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 23:14:04.905979 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":115,"threads_started":1}
I20260809 23:14:04.906579 4924 heartbeater.cc:347] Connected to a master server at 127.2.76.254:39573
I20260809 23:14:04.906664 4924 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:04.906808 4924 heartbeater.cc:511] Master 127.2.76.254:39573 requested a full tablet report, sending...
I20260809 23:14:04.907184 4294 ts_manager.cc:194] Registered new tserver with Master: cd23852b8fb146bea7eba9caac34cdd5 (127.2.76.194:44373)
I20260809 23:14:04.907603 4294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:38463
I20260809 23:14:04.910778 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 4683
I20260809 23:14:04.910877 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:14:04.911055 2355 external_mini_cluster.cc:1428] Setting key 0a0736e71266eeb0d1cbcecc48f397be
I20260809 23:14:04.912389 4375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:04.907556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:39401 (local address 127.2.76.254:39573)
0809 23:14:04.907592 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:04.907594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:04.907671 (+ 77us) server_negotiation.cc:407] Connection header received
0809 23:14:04.907714 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:04.907715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:04.907724 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:04.907758 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:04.908077 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.908524 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.909162 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.909345 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.909412 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:04.910146 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:04.910147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:04.910147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:04.910154 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:04.911312 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:04.911658 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:04.911660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:04.911660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:04.911719 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:04.911923 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:04.911924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:04.911924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:04.912036 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:04.912081 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:04.912284 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 23:14:04.912337 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:14:04.912549 4933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:04.907474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:39573 (local address 127.2.76.194:39401)
0809 23:14:04.907597 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:04.907607 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:04.907666 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:04.907797 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:04.907798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:04.907948 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:04.908003 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:04.908004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.908615 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.908616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:04.909084 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:04.909085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:04.909414 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:04.909415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:04.909492 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:04.909804 (+ 312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:04.909808 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:04.910064 (+ 256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:04.911400 (+ 1336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:04.911401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:04.911401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:04.911588 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:04.911791 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:04.911792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:04.911792 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:04.911859 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:04.912110 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:04.912111 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:04.912187 (+ 76us) client_negotiation.cc:769] Sending connection context
0809 23:14:04.912266 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 23:14:04.912379 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:14:04.934990 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:39573
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:36919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:05.011458 4937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:05.011631 4937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:05.011652 4937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:05.012802 4937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:05.012843 4937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:05.012862 4937 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:14:05.012913 4937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:14:05.014264 4937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:36919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.76.254:39573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:05.014495 4937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:05.014711 4937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:05.019122 4947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:05.019148 4945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:05.019121 4951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:05.019310 4937 server_base.cc:1034] running on GCE node
I20260809 23:14:05.019549 4937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:05.019809 4937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:05.020975 4937 hybrid_clock.cc:648] HybridClock initialized: now 1786317245020961 us; error 31 us; skew 500 ppm
Aug 09 23:14:05 dist-test-slave-7vjv krb5kdc[4216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317245, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:05.023159 4937 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:14:05.023633 4937 webserver.cc:460] Webserver started at http://127.2.76.195:42709/ using document root <none> and password file <none>
I20260809 23:14:05.023834 4937 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:05.023893 4937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:05.024022 4937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:05.024871 4937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "34c4895b47e046d58b1f0bf95fc58f8f"
format_stamp: "Formatted at 2026-08-09 23:14:05 on dist-test-slave-7vjv"
server_key: "93e8c746305ff96a7f969524e54d9d5c"
server_key_iv: "859da4bb8c47128d2f3d2962e5d291e5"
server_key_version: "encryptionkey@0"
I20260809 23:14:05.025195 4937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "34c4895b47e046d58b1f0bf95fc58f8f"
format_stamp: "Formatted at 2026-08-09 23:14:05 on dist-test-slave-7vjv"
server_key: "93e8c746305ff96a7f969524e54d9d5c"
server_key_iv: "859da4bb8c47128d2f3d2962e5d291e5"
server_key_version: "encryptionkey@0"
I20260809 23:14:05.026314 4937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:14:05.026911 4964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:05.027066 4937 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:05.027155 4937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "34c4895b47e046d58b1f0bf95fc58f8f"
format_stamp: "Formatted at 2026-08-09 23:14:05 on dist-test-slave-7vjv"
server_key: "93e8c746305ff96a7f969524e54d9d5c"
server_key_iv: "859da4bb8c47128d2f3d2962e5d291e5"
server_key_version: "encryptionkey@0"
I20260809 23:14:05.027232 4937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:05.040940 4937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:05.042603 4937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:05.042743 4937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:05.043269 4937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:05.043329 4937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:05.043366 4937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:05.043388 4937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:14:05 dist-test-slave-7vjv krb5kdc[4216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317245, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:05.053489 4937 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:46061
I20260809 23:14:05.053777 4937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:14:05.054260 5175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:46061 every 8 connection(s)
I20260809 23:14:05.054774 4375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.045693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:60645 (local address 127.2.76.254:39573)
0809 23:14:05.045773 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.045775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.045783 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:05.045804 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.045805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.045814 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.045897 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:05.046494 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.047090 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.048195 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.048394 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.048466 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.051899 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:05.051901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:05.051902 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:05.051911 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:05.053411 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:05.053868 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:05.053869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:05.053869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:05.053928 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:05.054207 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:05.054208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:05.054208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:05.054357 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:05.054417 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.054656 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.054714 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 23:14:05.054886 4986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.044266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:39573 (local address 127.2.76.195:60645)
0809 23:14:05.044438 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:05.044472 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:05.044979 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:05.046010 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:05.046013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:05.046243 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:05.046398 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.046401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.047185 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.047186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.048028 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.048031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.048502 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.048503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.048612 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.049082 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:05.049093 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:05.051725 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:05.053522 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:05.053529 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:05.053531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:05.053752 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:05.053999 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:05.054000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:05.054000 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:05.054133 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:05.054430 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:05.054441 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:05.054544 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 23:14:05.054627 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 23:14:05.054731 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":63,"threads_started":1}
I20260809 23:14:05.055393 5176 heartbeater.cc:347] Connected to a master server at 127.2.76.254:39573
I20260809 23:14:05.055469 5176 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:05.055639 5176 heartbeater.cc:511] Master 127.2.76.254:39573 requested a full tablet report, sending...
I20260809 23:14:05.056001 4293 ts_manager.cc:194] Registered new tserver with Master: 34c4895b47e046d58b1f0bf95fc58f8f (127.2.76.195:46061)
I20260809 23:14:05.056478 4293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:60645
I20260809 23:14:05.059325 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 4937
I20260809 23:14:05.059425 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:14:05.059585 2355 external_mini_cluster.cc:1428] Setting key b9c2ed6c1a75d34055bcbf0ecf67b776
I20260809 23:14:05.060427 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:14:05.063252 5184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.056353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:39573 (local address 127.2.76.195:41609)
0809 23:14:05.056494 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:05.056511 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:05.056576 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:05.056693 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:05.056693 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:05.056882 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:05.056941 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.056943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.057486 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.057486 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.057923 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.057924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.058207 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.058208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.058255 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.058546 (+ 291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:05.058549 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:05.058774 (+ 225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:05.062408 (+ 3634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:05.062409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:05.062410 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:05.062567 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:05.062781 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:05.062782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:05.062782 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:05.062823 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:05.063075 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:05.063076 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:05.063103 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 23:14:05.063140 (+ 37us) client_negotiation.cc:240] Negotiation successful
0809 23:14:05.063196 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":62,"threads_started":1}
I20260809 23:14:05.063333 4375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.056414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:41609 (local address 127.2.76.254:39573)
0809 23:14:05.056452 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.056454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.056565 (+ 111us) server_negotiation.cc:407] Connection header received
0809 23:14:05.056628 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.056629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.056638 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.056686 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:05.057017 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.057405 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.058000 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.058143 (+ 143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.058193 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.058839 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:05.058840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:05.058840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:05.058845 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:05.060370 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:05.062643 (+ 2273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:05.062644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:05.062644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:05.062711 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:05.062860 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:05.062860 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:05.062874 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:05.063021 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:05.063089 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.063167 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.063235 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 23:14:05 dist-test-slave-7vjv krb5kdc[4216](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317245, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 23:14:05 dist-test-slave-7vjv krb5kdc[4216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317245, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:05.083369 4375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.077278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52000 (local address 127.2.76.254:39573)
0809 23:14:05.077334 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.077335 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.077343 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:05.077396 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.077397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.077404 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.077457 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:05.077896 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.078302 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.078969 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.079140 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.079187 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.081216 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:05.081217 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:05.081218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:05.081224 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:05.082304 (+ 1080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:05.082626 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:05.082627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:05.082628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:05.082682 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:05.082883 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:05.082884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:05.082885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:05.083004 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:05.083110 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.083213 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.083282 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 23:14:05.084698 4293 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:14:05.086048 4293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:14:05.091570 5200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.087995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:36215 (local address 127.0.0.1:49568)
0809 23:14:05.088298 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:05.088305 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:05.088342 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:05.088728 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:05.088731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:05.088732 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:05.088786 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.088787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.089639 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.089640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.090558 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.090560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.091300 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.091301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.091369 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.091381 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 23:14:05.091459 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 23:14:05.091505 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":52,"thread_start_us":79,"threads_started":1}
I20260809 23:14:05.091596 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.088377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49568 (local address 127.2.76.193:36215)
0809 23:14:05.088581 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.088584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.088593 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:14:05.088622 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.088624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.088634 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.088717 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:05.088840 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.089549 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.090656 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.091224 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.091299 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.091348 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.091473 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.091511 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":126,"threads_started":1}
I20260809 23:14:05.091925 5195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.087986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:46061 (local address 127.0.0.1:42248)
0809 23:14:05.088222 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:05.088231 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:05.088279 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:05.088792 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:05.088794 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:05.088796 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:05.088848 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.088850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.089673 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.089675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.090872 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.090875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.091701 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.091702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.091753 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.091756 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:14:05.091808 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 23:14:05.091837 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":28,"thread_start_us":61,"threads_started":1}
I20260809 23:14:05.092231 5202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.088205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42248 (local address 127.2.76.195:46061)
0809 23:14:05.088591 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.088594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.088602 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:05.088626 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.088646 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.088659 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.088918 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:05.088978 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.089590 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.090957 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.091571 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.091630 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.091670 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.091931 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.092045 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":67,"threads_started":1}
I20260809 23:14:05.092433 4533 tablet_service.cc:1467] Processing CreateTablet for tablet cc158e305d1e4bdfa21de94aab8bb2c3 (DEFAULT_TABLE table=test-table [id=5ad4d6500fd24ea7bc7c3815a7916709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:05.092788 4533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc158e305d1e4bdfa21de94aab8bb2c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:05.092749 5061 tablet_service.cc:1467] Processing CreateTablet for tablet cc158e305d1e4bdfa21de94aab8bb2c3 (DEFAULT_TABLE table=test-table [id=5ad4d6500fd24ea7bc7c3815a7916709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:05.093019 5196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.087984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:44373 (local address 127.0.0.1:48780)
0809 23:14:05.088106 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:05.088140 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:05.088198 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:05.089918 (+ 1720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:05.089919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:05.089921 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:05.089955 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.089957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.091043 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.091043 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.091949 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.091952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.092833 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.092834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.092890 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.092894 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:14:05.092925 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 23:14:05.092962 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":44,"threads_started":1}
I20260809 23:14:05.093024 5061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc158e305d1e4bdfa21de94aab8bb2c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:05.093333 5203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.088247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48780 (local address 127.2.76.194:44373)
0809 23:14:05.089761 (+ 1514us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.089765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.089774 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:14:05.089808 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.089810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.089824 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.089895 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:05.090021 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.090966 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.092045 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.092731 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.093123 (+ 392us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.093186 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.093209 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.093247 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1476,"thread_start_us":72,"threads_started":1}
I20260809 23:14:05.095417 5205 tablet_bootstrap.cc:492] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f: Bootstrap starting.
I20260809 23:14:05.095841 5206 tablet_bootstrap.cc:492] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9: Bootstrap starting.
I20260809 23:14:05.095983 4803 tablet_service.cc:1467] Processing CreateTablet for tablet cc158e305d1e4bdfa21de94aab8bb2c3 (DEFAULT_TABLE table=test-table [id=5ad4d6500fd24ea7bc7c3815a7916709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:05.096329 4803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc158e305d1e4bdfa21de94aab8bb2c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:05.096513 5205 tablet_bootstrap.cc:654] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:05.096527 5206 tablet_bootstrap.cc:654] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:05.096901 5206 log.cc:824] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:05.096905 5205 log.cc:824] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:05.097640 5205 tablet_bootstrap.cc:492] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f: No bootstrap required, opened a new log
I20260809 23:14:05.097721 5205 ts_tablet_manager.cc:1397] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:05.097719 5206 tablet_bootstrap.cc:492] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9: No bootstrap required, opened a new log
I20260809 23:14:05.097783 5206 ts_tablet_manager.cc:1397] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:14:05.099013 5205 raft_consensus.cc:348] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } }
I20260809 23:14:05.099166 5205 raft_consensus.cc:374] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:05.099205 5205 raft_consensus.cc:729] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c4895b47e046d58b1f0bf95fc58f8f, State: Initialized, Role: FOLLOWER
I20260809 23:14:05.099294 5205 consensus_queue.cc:260] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } }
I20260809 23:14:05.099515 5205 ts_tablet_manager.cc:1428] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:05.099581 5176 heartbeater.cc:503] Master 127.2.76.254:39573 was elected leader, sending a full tablet report...
I20260809 23:14:05.100011 5220 tablet_bootstrap.cc:492] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5: Bootstrap starting.
I20260809 23:14:05.100251 5206 raft_consensus.cc:348] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } }
I20260809 23:14:05.100405 5206 raft_consensus.cc:374] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:05.100549 5206 raft_consensus.cc:729] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c201a739bce54c23b7add3aa5e9847f9, State: Initialized, Role: FOLLOWER
I20260809 23:14:05.100625 5220 tablet_bootstrap.cc:654] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:05.100692 5206 consensus_queue.cc:260] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } }
I20260809 23:14:05.101064 5220 log.cc:824] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:05.101137 5206 ts_tablet_manager.cc:1428] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 23:14:05.101238 4662 heartbeater.cc:503] Master 127.2.76.254:39573 was elected leader, sending a full tablet report...
I20260809 23:14:05.101936 5220 tablet_bootstrap.cc:492] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5: No bootstrap required, opened a new log
I20260809 23:14:05.101995 5220 ts_tablet_manager.cc:1397] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:14:05.103365 5220 raft_consensus.cc:348] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } }
I20260809 23:14:05.103467 5220 raft_consensus.cc:374] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:05.103485 5220 raft_consensus.cc:729] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd23852b8fb146bea7eba9caac34cdd5, State: Initialized, Role: FOLLOWER
I20260809 23:14:05.103603 5220 consensus_queue.cc:260] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } }
I20260809 23:14:05.103824 5220 ts_tablet_manager.cc:1428] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 23:14:05.103883 4924 heartbeater.cc:503] Master 127.2.76.254:39573 was elected leader, sending a full tablet report...
W20260809 23:14:05.154779 4925 tablet.cc:2369] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:14:05.220518 5221 raft_consensus.cc:482] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:14:05.220628 5221 raft_consensus.cc:504] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } }
I20260809 23:14:05.220932 5221 leader_election.cc:288] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34c4895b47e046d58b1f0bf95fc58f8f (127.2.76.195:46061), cd23852b8fb146bea7eba9caac34cdd5 (127.2.76.194:44373)
I20260809 23:14:05.224401 5203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.221186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:40523 (local address 127.2.76.194:44373)
0809 23:14:05.221226 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.221228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.221389 (+ 161us) server_negotiation.cc:407] Connection header received
0809 23:14:05.221417 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.221418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.221427 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.221486 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:05.221614 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.222227 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.223598 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.224057 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.224142 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.224149 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.224305 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.224339 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:14:05.224401 5202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.221163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:37891 (local address 127.2.76.195:46061)
0809 23:14:05.221214 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.221216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.221226 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:14:05.221251 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.221252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.221262 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.221319 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:05.221492 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.222227 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.223598 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.224063 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.224139 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.224147 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.224304 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.224340 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 23:14:05.224414 4461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.221093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:46061 (local address 127.2.76.193:37891)
0809 23:14:05.221148 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:05.221158 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:05.221198 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:05.221343 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:05.221344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:05.221346 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:05.221415 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.221417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.222304 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.222305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.223517 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.223518 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.224150 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.224151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.224234 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.224238 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:14:05.224286 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 23:14:05.224329 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":66}
I20260809 23:14:05.224426 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.221112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:44373 (local address 127.2.76.193:40523)
0809 23:14:05.221293 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:05.221317 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:05.221368 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:05.221490 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:05.221490 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:05.221492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:05.221548 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.221549 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.222304 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.222305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.223515 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.223518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.224143 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.224145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.224234 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.224238 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:14:05.224286 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 23:14:05.224329 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":98,"thread_start_us":109,"threads_started":1}
I20260809 23:14:05.224853 5100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc158e305d1e4bdfa21de94aab8bb2c3" candidate_uuid: "c201a739bce54c23b7add3aa5e9847f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" is_pre_election: true
I20260809 23:14:05.224925 4840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc158e305d1e4bdfa21de94aab8bb2c3" candidate_uuid: "c201a739bce54c23b7add3aa5e9847f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd23852b8fb146bea7eba9caac34cdd5" is_pre_election: true
I20260809 23:14:05.224979 5100 raft_consensus.cc:2454] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c201a739bce54c23b7add3aa5e9847f9 in term 0.
I20260809 23:14:05.225106 4840 raft_consensus.cc:2454] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c201a739bce54c23b7add3aa5e9847f9 in term 0.
I20260809 23:14:05.225239 4447 leader_election.cc:302] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34c4895b47e046d58b1f0bf95fc58f8f, c201a739bce54c23b7add3aa5e9847f9; no voters:
I20260809 23:14:05.225363 5221 raft_consensus.cc:2781] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:14:05.225425 5221 raft_consensus.cc:482] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:14:05.225451 5221 raft_consensus.cc:3037] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:05.226336 5221 raft_consensus.cc:504] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } }
I20260809 23:14:05.226473 5221 leader_election.cc:288] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [CANDIDATE]: Term 1 election: Requested vote from peers 34c4895b47e046d58b1f0bf95fc58f8f (127.2.76.195:46061), cd23852b8fb146bea7eba9caac34cdd5 (127.2.76.194:44373)
I20260809 23:14:05.226621 4840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc158e305d1e4bdfa21de94aab8bb2c3" candidate_uuid: "c201a739bce54c23b7add3aa5e9847f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd23852b8fb146bea7eba9caac34cdd5"
I20260809 23:14:05.226635 5100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc158e305d1e4bdfa21de94aab8bb2c3" candidate_uuid: "c201a739bce54c23b7add3aa5e9847f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34c4895b47e046d58b1f0bf95fc58f8f"
I20260809 23:14:05.226703 4840 raft_consensus.cc:3037] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:05.226701 5100 raft_consensus.cc:3037] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:05.227325 5100 raft_consensus.cc:2454] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c201a739bce54c23b7add3aa5e9847f9 in term 1.
I20260809 23:14:05.227470 4840 raft_consensus.cc:2454] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c201a739bce54c23b7add3aa5e9847f9 in term 1.
I20260809 23:14:05.227475 4447 leader_election.cc:302] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34c4895b47e046d58b1f0bf95fc58f8f, c201a739bce54c23b7add3aa5e9847f9; no voters:
I20260809 23:14:05.227672 5221 raft_consensus.cc:2781] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:05.227835 5221 raft_consensus.cc:686] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [term 1 LEADER]: Becoming Leader. State: Replica: c201a739bce54c23b7add3aa5e9847f9, State: Running, Role: LEADER
I20260809 23:14:05.227931 5221 consensus_queue.cc:237] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } }
I20260809 23:14:05.228720 4294 catalog_manager.cc:5495] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c201a739bce54c23b7add3aa5e9847f9 (127.2.76.193). New cstate: current_term: 1 leader_uuid: "c201a739bce54c23b7add3aa5e9847f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } health_report { overall_health: HEALTHY } } }
I20260809 23:14:05.236145 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.233824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49574 (local address 127.2.76.193:36215)
0809 23:14:05.233883 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.233884 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.233893 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:14:05.233968 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.233969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.233984 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.234066 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:14:05.234208 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.234614 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.235359 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.235544 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.235612 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.235810 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:14:05.235894 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:14:05.235967 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.236019 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.236037 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:14:05.237680 5100 raft_consensus.cc:1261] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f [term 1 FOLLOWER]: Refusing update from remote peer c201a739bce54c23b7add3aa5e9847f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:05.237680 4840 raft_consensus.cc:1261] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5 [term 1 FOLLOWER]: Refusing update from remote peer c201a739bce54c23b7add3aa5e9847f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:05.238080 5221 consensus_queue.cc:1035] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:05.240756 5221 consensus_queue.cc:1035] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:05.244272 4294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:05.244359 4294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:05.245057 4294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d: Sending DeleteTablet for 3 replicas of tablet cc158e305d1e4bdfa21de94aab8bb2c3
I20260809 23:14:05.245437 4803 tablet_service.cc:1514] Processing DeleteTablet for tablet cc158e305d1e4bdfa21de94aab8bb2c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:05 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:48780
I20260809 23:14:05.245421 5061 tablet_service.cc:1514] Processing DeleteTablet for tablet cc158e305d1e4bdfa21de94aab8bb2c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:05 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:42248
I20260809 23:14:05.245446 4533 tablet_service.cc:1514] Processing DeleteTablet for tablet cc158e305d1e4bdfa21de94aab8bb2c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:05 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:49568
I20260809 23:14:05.245659 5243 tablet_replica.cc:330] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9: stopping tablet replica
I20260809 23:14:05.245747 5243 raft_consensus.cc:2229] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:14:05.245852 5243 raft_consensus.cc:2258] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:05.245967 5249 tablet_replica.cc:330] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f: stopping tablet replica
I20260809 23:14:05.245978 4294 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 23:14:05.246058 5249 raft_consensus.cc:2229] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 23:14:05.246106 4294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:14:05.246147 5249 raft_consensus.cc:2258] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:05.246351 5243 ts_tablet_manager.cc:1905] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:05.247344 5249 ts_tablet_manager.cc:1905] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:05.247375 5061 tablet_service.cc:1467] Processing CreateTablet for tablet 4ec72832c2ab4d869e63b2b899d2380c (DEFAULT_TABLE table=test-table [id=08f3579d4682491faa4820f5fa146af1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:05.247470 5061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ec72832c2ab4d869e63b2b899d2380c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:05.248052 5243 ts_tablet_manager.cc:1918] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:14:05.248164 5243 log.cc:1196] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/wal/wals/cc158e305d1e4bdfa21de94aab8bb2c3
I20260809 23:14:05.248394 5243 ts_tablet_manager.cc:1939] T cc158e305d1e4bdfa21de94aab8bb2c3 P c201a739bce54c23b7add3aa5e9847f9: Deleting consensus metadata
I20260809 23:14:05.248894 5205 tablet_bootstrap.cc:492] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f: Bootstrap starting.
I20260809 23:14:05.248996 4271 catalog_manager.cc:4841] TS c201a739bce54c23b7add3aa5e9847f9 (127.2.76.193:36215): tablet cc158e305d1e4bdfa21de94aab8bb2c3 (table test-table [id=5ad4d6500fd24ea7bc7c3815a7916709]) successfully deleted
I20260809 23:14:05.249316 5205 tablet_bootstrap.cc:654] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:05.249329 4533 tablet_service.cc:1467] Processing CreateTablet for tablet 4ec72832c2ab4d869e63b2b899d2380c (DEFAULT_TABLE table=test-table [id=08f3579d4682491faa4820f5fa146af1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:05.249451 4533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ec72832c2ab4d869e63b2b899d2380c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:05.249590 5249 ts_tablet_manager.cc:1918] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:14:05.249650 5249 log.cc:1196] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/cc158e305d1e4bdfa21de94aab8bb2c3
I20260809 23:14:05.249889 5249 ts_tablet_manager.cc:1939] T cc158e305d1e4bdfa21de94aab8bb2c3 P 34c4895b47e046d58b1f0bf95fc58f8f: Deleting consensus metadata
I20260809 23:14:05.250216 5250 tablet_replica.cc:330] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5: stopping tablet replica
I20260809 23:14:05.250305 5250 raft_consensus.cc:2229] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:05.250403 5250 raft_consensus.cc:2258] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:05.250794 5206 tablet_bootstrap.cc:492] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9: Bootstrap starting.
I20260809 23:14:05.250787 5250 ts_tablet_manager.cc:1905] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:05.251242 5206 tablet_bootstrap.cc:654] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:05.251804 4266 catalog_manager.cc:4841] TS 34c4895b47e046d58b1f0bf95fc58f8f (127.2.76.195:46061): tablet cc158e305d1e4bdfa21de94aab8bb2c3 (table test-table [id=5ad4d6500fd24ea7bc7c3815a7916709]) successfully deleted
I20260809 23:14:05.252046 5206 tablet_bootstrap.cc:492] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9: No bootstrap required, opened a new log
I20260809 23:14:05.252089 5206 ts_tablet_manager.cc:1397] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 23:14:05.252279 5206 raft_consensus.cc:348] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.252347 5206 raft_consensus.cc:374] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:05.252367 5206 raft_consensus.cc:729] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c201a739bce54c23b7add3aa5e9847f9, State: Initialized, Role: FOLLOWER
I20260809 23:14:05.252400 5206 consensus_queue.cc:260] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.252476 5206 ts_tablet_manager.cc:1428] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:05.252478 4803 tablet_service.cc:1467] Processing CreateTablet for tablet 4ec72832c2ab4d869e63b2b899d2380c (DEFAULT_TABLE table=test-table [id=08f3579d4682491faa4820f5fa146af1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:05.252565 4803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ec72832c2ab4d869e63b2b899d2380c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:05.252918 5250 ts_tablet_manager.cc:1918] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:14:05.252974 5250 log.cc:1196] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/cc158e305d1e4bdfa21de94aab8bb2c3
I20260809 23:14:05.253233 5250 ts_tablet_manager.cc:1939] T cc158e305d1e4bdfa21de94aab8bb2c3 P cd23852b8fb146bea7eba9caac34cdd5: Deleting consensus metadata
I20260809 23:14:05.253619 4268 catalog_manager.cc:4841] TS cd23852b8fb146bea7eba9caac34cdd5 (127.2.76.194:44373): tablet cc158e305d1e4bdfa21de94aab8bb2c3 (table test-table [id=5ad4d6500fd24ea7bc7c3815a7916709]) successfully deleted
I20260809 23:14:05.253732 5220 tablet_bootstrap.cc:492] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5: Bootstrap starting.
I20260809 23:14:05.254117 5220 tablet_bootstrap.cc:654] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:05.254861 5220 tablet_bootstrap.cc:492] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5: No bootstrap required, opened a new log
I20260809 23:14:05.254925 5220 ts_tablet_manager.cc:1397] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 23:14:05.255057 5220 raft_consensus.cc:348] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.255120 5220 raft_consensus.cc:374] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:05.255144 5220 raft_consensus.cc:729] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd23852b8fb146bea7eba9caac34cdd5, State: Initialized, Role: FOLLOWER
I20260809 23:14:05.255187 5220 consensus_queue.cc:260] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.255285 5220 ts_tablet_manager.cc:1428] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 23:14:05.256611 5205 tablet_bootstrap.cc:492] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f: No bootstrap required, opened a new log
I20260809 23:14:05.256665 5205 ts_tablet_manager.cc:1397] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260809 23:14:05.256795 5205 raft_consensus.cc:348] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.256862 5205 raft_consensus.cc:374] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:05.256882 5205 raft_consensus.cc:729] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c4895b47e046d58b1f0bf95fc58f8f, State: Initialized, Role: FOLLOWER
I20260809 23:14:05.256918 5205 consensus_queue.cc:260] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.257014 5205 ts_tablet_manager.cc:1428] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260809 23:14:05.268697 4666 tablet.cc:2369] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:14:05.304355 5177 tablet.cc:2369] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:14:05.457798 5224 raft_consensus.cc:482] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:14:05.457891 5224 raft_consensus.cc:504] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.458258 5224 leader_election.cc:288] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c201a739bce54c23b7add3aa5e9847f9 (127.2.76.193:36215), 34c4895b47e046d58b1f0bf95fc58f8f (127.2.76.195:46061)
I20260809 23:14:05.461046 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.458443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:46061 (local address 127.2.76.194:55523)
0809 23:14:05.458572 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:05.458583 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:05.458673 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:05.458807 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:05.458808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:05.458810 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:05.458907 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.458908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.459484 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.459485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.460301 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.460303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.460787 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.460789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.460852 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.460857 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:14:05.460905 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 23:14:05.460932 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":58,"thread_start_us":46,"threads_started":1}
I20260809 23:14:05.461037 5202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.458509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:55523 (local address 127.2.76.195:46061)
0809 23:14:05.458565 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.458567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.458675 (+ 108us) server_negotiation.cc:407] Connection header received
0809 23:14:05.458732 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.458733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.458740 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.458801 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:05.458990 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.459399 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.460377 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.460711 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.460764 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.460771 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.460927 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.460972 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 23:14:05.461108 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.458531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:37673 (local address 127.2.76.193:36215)
0809 23:14:05.458576 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.458578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.458647 (+ 69us) server_negotiation.cc:407] Connection header received
0809 23:14:05.458716 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.458717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.458727 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.458810 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:05.458962 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.459397 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.460409 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.460738 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.460826 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.460834 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.460996 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.461034 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:14:05.461158 5260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.458443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:36215 (local address 127.2.76.194:37673)
0809 23:14:05.458560 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:05.458572 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:05.458643 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:05.458817 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:05.458818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:05.458820 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:05.458899 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.458900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.459475 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.459476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.460306 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.460309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.460838 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.460839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.460915 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.460920 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:14:05.460972 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 23:14:05.461016 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":43,"thread_start_us":63,"threads_started":1}
I20260809 23:14:05.461483 5100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ec72832c2ab4d869e63b2b899d2380c" candidate_uuid: "cd23852b8fb146bea7eba9caac34cdd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" is_pre_election: true
I20260809 23:14:05.461596 5100 raft_consensus.cc:2454] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd23852b8fb146bea7eba9caac34cdd5 in term 0.
I20260809 23:14:05.461586 4584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ec72832c2ab4d869e63b2b899d2380c" candidate_uuid: "cd23852b8fb146bea7eba9caac34cdd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c201a739bce54c23b7add3aa5e9847f9" is_pre_election: true
I20260809 23:14:05.461751 4584 raft_consensus.cc:2454] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd23852b8fb146bea7eba9caac34cdd5 in term 0.
I20260809 23:14:05.461812 4709 leader_election.cc:302] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34c4895b47e046d58b1f0bf95fc58f8f, cd23852b8fb146bea7eba9caac34cdd5; no voters:
I20260809 23:14:05.461971 5224 raft_consensus.cc:2781] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:14:05.462013 5224 raft_consensus.cc:482] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:14:05.462034 5224 raft_consensus.cc:3037] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:05.462777 5224 raft_consensus.cc:504] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.462901 5224 leader_election.cc:288] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [CANDIDATE]: Term 1 election: Requested vote from peers c201a739bce54c23b7add3aa5e9847f9 (127.2.76.193:36215), 34c4895b47e046d58b1f0bf95fc58f8f (127.2.76.195:46061)
I20260809 23:14:05.463025 4584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ec72832c2ab4d869e63b2b899d2380c" candidate_uuid: "cd23852b8fb146bea7eba9caac34cdd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c201a739bce54c23b7add3aa5e9847f9"
I20260809 23:14:05.463047 5100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ec72832c2ab4d869e63b2b899d2380c" candidate_uuid: "cd23852b8fb146bea7eba9caac34cdd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34c4895b47e046d58b1f0bf95fc58f8f"
I20260809 23:14:05.463121 5100 raft_consensus.cc:3037] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:05.463124 4584 raft_consensus.cc:3037] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:05.463722 5100 raft_consensus.cc:2454] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd23852b8fb146bea7eba9caac34cdd5 in term 1.
I20260809 23:14:05.463826 4584 raft_consensus.cc:2454] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd23852b8fb146bea7eba9caac34cdd5 in term 1.
I20260809 23:14:05.463896 4709 leader_election.cc:302] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34c4895b47e046d58b1f0bf95fc58f8f, cd23852b8fb146bea7eba9caac34cdd5; no voters:
I20260809 23:14:05.463985 5224 raft_consensus.cc:2781] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:05.464080 5224 raft_consensus.cc:686] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [term 1 LEADER]: Becoming Leader. State: Replica: cd23852b8fb146bea7eba9caac34cdd5, State: Running, Role: LEADER
I20260809 23:14:05.464159 5224 consensus_queue.cc:237] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.464758 4294 catalog_manager.cc:5495] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd23852b8fb146bea7eba9caac34cdd5 (127.2.76.194). New cstate: current_term: 1 leader_uuid: "cd23852b8fb146bea7eba9caac34cdd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } health_report { overall_health: UNKNOWN } } }
Aug 09 23:14:05 dist-test-slave-7vjv krb5kdc[4216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317244, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:05.514879 5273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.508675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:39573 (local address 127.0.0.1:52006)
0809 23:14:05.508896 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:05.508905 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:05.508986 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:05.509140 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:05.509141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:05.509432 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:05.509481 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.509484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.510073 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.510074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.510550 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.510551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.510858 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.510859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.510909 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.511444 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:05.511448 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:05.512809 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:05.514066 (+ 1257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:05.514072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:05.514074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:05.514249 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:05.514410 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:05.514411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:05.514411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:05.514461 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:05.514659 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:05.514664 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:05.514700 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 23:14:05.514745 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 23:14:05.514819 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":119,"threads_started":1}
I20260809 23:14:05.514932 4375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.508746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52006 (local address 127.2.76.254:39573)
0809 23:14:05.508860 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.508862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.508975 (+ 113us) server_negotiation.cc:407] Connection header received
0809 23:14:05.509057 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.509057 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.509068 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.509135 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:05.509562 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.509986 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.510627 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.510793 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.510859 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.512894 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:05.512895 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:05.512896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:05.512902 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:05.513999 (+ 1097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:05.514311 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:05.514311 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:05.514312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:05.514358 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:05.514516 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:05.514517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:05.514517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:05.514613 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:05.514656 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.514774 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.514860 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":88}
I20260809 23:14:05.516855 4294 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:52006:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 23:14:05.517083 4294 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:14:05.518424 5061 tablet_service.cc:1467] Processing CreateTablet for tablet ee1cc29564e54ecb82893b020768aee9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c73478c52e9f43489e8a43af771c90c5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:14:05.518394 4803 tablet_service.cc:1467] Processing CreateTablet for tablet ee1cc29564e54ecb82893b020768aee9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c73478c52e9f43489e8a43af771c90c5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:14:05.518565 5061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee1cc29564e54ecb82893b020768aee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:05.518597 4803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee1cc29564e54ecb82893b020768aee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:05.519709 5205 tablet_bootstrap.cc:492] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f: Bootstrap starting.
I20260809 23:14:05.520040 5205 tablet_bootstrap.cc:654] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:05.520113 5220 tablet_bootstrap.cc:492] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5: Bootstrap starting.
I20260809 23:14:05.518391 4533 tablet_service.cc:1467] Processing CreateTablet for tablet ee1cc29564e54ecb82893b020768aee9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c73478c52e9f43489e8a43af771c90c5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:14:05.520650 5220 tablet_bootstrap.cc:654] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:05.520715 4533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee1cc29564e54ecb82893b020768aee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:05.520818 5205 tablet_bootstrap.cc:492] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f: No bootstrap required, opened a new log
I20260809 23:14:05.520876 5205 ts_tablet_manager.cc:1397] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 23:14:05.521054 5205 raft_consensus.cc:348] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.521106 5205 raft_consensus.cc:374] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:05.521122 5205 raft_consensus.cc:729] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c4895b47e046d58b1f0bf95fc58f8f, State: Initialized, Role: FOLLOWER
I20260809 23:14:05.521157 5205 consensus_queue.cc:260] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.521224 5205 ts_tablet_manager.cc:1428] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:05.521337 5218 tablet_replica.cc:439] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } }
I20260809 23:14:05.521409 5218 tablet_replica.cc:442] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:14:05.521437 5220 tablet_bootstrap.cc:492] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5: No bootstrap required, opened a new log
I20260809 23:14:05.521481 5220 ts_tablet_manager.cc:1397] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:05.521629 5220 raft_consensus.cc:348] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.521721 5220 raft_consensus.cc:374] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:05.521745 5220 raft_consensus.cc:729] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd23852b8fb146bea7eba9caac34cdd5, State: Initialized, Role: FOLLOWER
I20260809 23:14:05.521781 5220 consensus_queue.cc:260] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.521878 5220 ts_tablet_manager.cc:1428] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:05.521920 5224 tablet_replica.cc:439] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } }
I20260809 23:14:05.522017 5224 tablet_replica.cc:442] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:14:05.522275 5206 tablet_bootstrap.cc:492] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9: Bootstrap starting.
I20260809 23:14:05.522620 5206 tablet_bootstrap.cc:654] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:05.523175 5206 tablet_bootstrap.cc:492] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9: No bootstrap required, opened a new log
I20260809 23:14:05.523219 5206 ts_tablet_manager.cc:1397] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 23:14:05.523324 5206 raft_consensus.cc:348] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.523398 5206 raft_consensus.cc:374] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:05.523422 5206 raft_consensus.cc:729] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c201a739bce54c23b7add3aa5e9847f9, State: Initialized, Role: FOLLOWER
I20260809 23:14:05.523468 5206 consensus_queue.cc:260] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.523555 5206 ts_tablet_manager.cc:1428] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:05.523581 5221 tablet_replica.cc:439] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } }
I20260809 23:14:05.523687 5221 tablet_replica.cc:442] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:14:05.533836 5218 raft_consensus.cc:482] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:14:05.533885 5218 raft_consensus.cc:504] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.534199 5218 leader_election.cc:288] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c201a739bce54c23b7add3aa5e9847f9 (127.2.76.193:36215), cd23852b8fb146bea7eba9caac34cdd5 (127.2.76.194:44373)
I20260809 23:14:05.536863 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.534350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:37223 (local address 127.2.76.193:36215)
0809 23:14:05.534385 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.534386 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.534394 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:05.534442 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.534443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.534454 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.534510 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:05.534640 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.535054 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.536054 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.536591 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.536636 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.536642 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.536804 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.536826 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:14:05.536885 4986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.534305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:36215 (local address 127.2.76.195:37223)
0809 23:14:05.534340 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:05.534352 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:05.534388 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:05.534518 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:05.534519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:05.534520 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:05.534579 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.534581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.535120 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.535121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.535960 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.535963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.536668 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.536669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.536737 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.536740 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:14:05.536795 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 23:14:05.536826 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 23:14:05.537112 4584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee1cc29564e54ecb82893b020768aee9" candidate_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c201a739bce54c23b7add3aa5e9847f9" is_pre_election: true
I20260809 23:14:05.537190 4584 raft_consensus.cc:2454] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34c4895b47e046d58b1f0bf95fc58f8f in term 0.
I20260809 23:14:05.537312 5203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.534398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:55497 (local address 127.2.76.194:44373)
0809 23:14:05.534440 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.534441 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.534513 (+ 72us) server_negotiation.cc:407] Connection header received
0809 23:14:05.534534 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.534535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.534542 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.534594 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:05.534746 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.535136 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.536518 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.536982 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.537071 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.537079 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.537229 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.537262 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 23:14:05.537322 5282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.534333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:44373 (local address 127.2.76.195:55497)
0809 23:14:05.534433 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:05.534440 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:05.534494 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:05.534604 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:05.534604 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:05.534619 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:05.534687 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.534689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.535204 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.535205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.536433 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.536436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.537074 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.537075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.537145 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.537148 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:14:05.537222 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 23:14:05.537256 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":41,"threads_started":1}
I20260809 23:14:05.537367 4971 leader_election.cc:302] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34c4895b47e046d58b1f0bf95fc58f8f, c201a739bce54c23b7add3aa5e9847f9; no voters:
I20260809 23:14:05.537482 5218 raft_consensus.cc:2781] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:14:05.537532 5218 raft_consensus.cc:482] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:14:05.537557 5218 raft_consensus.cc:3037] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:05.537534 4840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee1cc29564e54ecb82893b020768aee9" candidate_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd23852b8fb146bea7eba9caac34cdd5" is_pre_election: true
I20260809 23:14:05.537597 4840 raft_consensus.cc:2454] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34c4895b47e046d58b1f0bf95fc58f8f in term 0.
I20260809 23:14:05.538141 5218 raft_consensus.cc:504] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.538285 5218 leader_election.cc:288] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [CANDIDATE]: Term 1 election: Requested vote from peers c201a739bce54c23b7add3aa5e9847f9 (127.2.76.193:36215), cd23852b8fb146bea7eba9caac34cdd5 (127.2.76.194:44373)
I20260809 23:14:05.538414 4584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee1cc29564e54ecb82893b020768aee9" candidate_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c201a739bce54c23b7add3aa5e9847f9"
I20260809 23:14:05.538493 4584 raft_consensus.cc:3037] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:05.538462 4840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee1cc29564e54ecb82893b020768aee9" candidate_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd23852b8fb146bea7eba9caac34cdd5"
I20260809 23:14:05.538547 4840 raft_consensus.cc:3037] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:05.538955 4584 raft_consensus.cc:2454] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34c4895b47e046d58b1f0bf95fc58f8f in term 1.
I20260809 23:14:05.539105 4971 leader_election.cc:302] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34c4895b47e046d58b1f0bf95fc58f8f, c201a739bce54c23b7add3aa5e9847f9; no voters:
I20260809 23:14:05.539136 4840 raft_consensus.cc:2454] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34c4895b47e046d58b1f0bf95fc58f8f in term 1.
I20260809 23:14:05.539194 5218 raft_consensus.cc:2781] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:05.539335 5218 raft_consensus.cc:686] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [term 1 LEADER]: Becoming Leader. State: Replica: 34c4895b47e046d58b1f0bf95fc58f8f, State: Running, Role: LEADER
I20260809 23:14:05.539393 5218 consensus_queue.cc:237] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } }
I20260809 23:14:05.539778 5283 tablet_replica.cc:439] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f: TxnStatusTablet state changed. Reason: New leader 34c4895b47e046d58b1f0bf95fc58f8f. Latest consensus state: current_term: 1 leader_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } }
I20260809 23:14:05.539855 5283 tablet_replica.cc:442] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:14:05.540014 4294 catalog_manager.cc:5495] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 34c4895b47e046d58b1f0bf95fc58f8f (127.2.76.195). New cstate: current_term: 1 leader_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } health_report { overall_health: HEALTHY } } }
I20260809 23:14:05.540212 5285 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 23:14:05.540313 5285 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 23:14:05.545358 5273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.542879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:46061 (local address 127.0.0.1:42254)
0809 23:14:05.542946 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:05.542955 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:05.542991 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:05.543280 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:05.543280 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:05.543281 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:14:05.543309 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.543311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.544024 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.544025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.544592 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.544594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.544978 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.544979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.545021 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.545033 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:14:05.545256 (+ 223us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:14:05.545262 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 23:14:05.545291 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 23:14:05.545318 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 23:14:05.545375 5202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.543099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42254 (local address 127.2.76.195:46061)
0809 23:14:05.543151 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.543153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.543161 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:05.543185 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.543186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.543201 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.543269 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:14:05.543351 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.543953 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.544666 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.544911 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.544977 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.545104 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:14:05.545198 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:14:05.545267 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.545308 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.545323 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 23:14:05.546455 4840 raft_consensus.cc:1261] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5 [term 1 FOLLOWER]: Refusing update from remote peer 34c4895b47e046d58b1f0bf95fc58f8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:05.546471 4584 raft_consensus.cc:1261] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9 [term 1 FOLLOWER]: Refusing update from remote peer 34c4895b47e046d58b1f0bf95fc58f8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:05.546706 5218 consensus_queue.cc:1035] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:05.546846 5218 consensus_queue.cc:1035] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:05.547153 5224 tablet_replica.cc:439] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5: TxnStatusTablet state changed. Reason: New leader 34c4895b47e046d58b1f0bf95fc58f8f. Latest consensus state: current_term: 1 leader_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } }
I20260809 23:14:05.547231 5224 tablet_replica.cc:442] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:14:05.547703 5221 tablet_replica.cc:439] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9: TxnStatusTablet state changed. Reason: New leader 34c4895b47e046d58b1f0bf95fc58f8f. Latest consensus state: current_term: 1 leader_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } }
I20260809 23:14:05.547798 5221 tablet_replica.cc:442] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:14:05.547763 5283 tablet_replica.cc:439] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } }
I20260809 23:14:05.547833 5283 tablet_replica.cc:442] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:14:05.547935 5237 mvcc.cc:205] Tried to move back new op lower bound from 7316755437756461056 to 7316755437729636352. Current Snapshot: MvccSnapshot[applied={T|T < 7316755437756461056}]
I20260809 23:14:05.548233 5218 tablet_replica.cc:439] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } }
I20260809 23:14:05.548336 5218 tablet_replica.cc:442] T ee1cc29564e54ecb82893b020768aee9 P 34c4895b47e046d58b1f0bf95fc58f8f: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:14:05.548342 5224 tablet_replica.cc:439] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } }
I20260809 23:14:05.548399 5232 mvcc.cc:205] Tried to move back new op lower bound from 7316755437756461056 to 7316755437729636352. Current Snapshot: MvccSnapshot[applied={T|T < 7316755437756461056}]
I20260809 23:14:05.548866 5231 mvcc.cc:205] Tried to move back new op lower bound from 7316755437756461056 to 7316755437729636352. Current Snapshot: MvccSnapshot[applied={T|T < 7316755437756461056}]
I20260809 23:14:05.548408 5224 tablet_replica.cc:442] T ee1cc29564e54ecb82893b020768aee9 P cd23852b8fb146bea7eba9caac34cdd5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:14:05.548942 5221 tablet_replica.cc:439] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 } } peers { permanent_uuid: "cd23852b8fb146bea7eba9caac34cdd5" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44373 } } peers { permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 } } }
I20260809 23:14:05.548995 5221 tablet_replica.cc:442] T ee1cc29564e54ecb82893b020768aee9 P c201a739bce54c23b7add3aa5e9847f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:14:05.552551 5203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.550208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48782 (local address 127.2.76.194:44373)
0809 23:14:05.550244 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.550246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.550254 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:05.550314 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.550314 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.550328 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.550374 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:14:05.550499 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.550991 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.551813 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.552012 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.552083 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.552270 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:14:05.552357 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:14:05.552431 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.552480 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.552494 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:14:05.557049 5202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.554651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:46649 (local address 127.2.76.195:46061)
0809 23:14:05.554710 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.554712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.554834 (+ 122us) server_negotiation.cc:407] Connection header received
0809 23:14:05.554884 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.554885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.554911 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.554961 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:14:05.555107 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.555455 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.556525 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.556682 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.556727 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.556902 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:14:05.556922 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:14:05.556951 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.556996 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.557013 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 23:14:05.557112 5295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.554583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:46061 (local address 127.2.76.194:46649)
0809 23:14:05.554780 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:05.554788 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:05.554841 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:05.554998 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:05.555000 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:05.555001 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:14:05.555058 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.555060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.555527 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.555528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.556441 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.556444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.556759 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.556760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.556816 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.556830 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:14:05.556952 (+ 122us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:14:05.556953 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 23:14:05.556994 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 23:14:05.557028 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":132,"threads_started":1}
I20260809 23:14:05.558856 4584 raft_consensus.cc:1261] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9 [term 1 FOLLOWER]: Refusing update from remote peer cd23852b8fb146bea7eba9caac34cdd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:05.558876 5100 raft_consensus.cc:1261] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f [term 1 FOLLOWER]: Refusing update from remote peer cd23852b8fb146bea7eba9caac34cdd5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:05.559085 5224 consensus_queue.cc:1035] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c201a739bce54c23b7add3aa5e9847f9" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:05.559196 5261 consensus_queue.cc:1035] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34c4895b47e046d58b1f0bf95fc58f8f" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:05.567099 5203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.564743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:36999 (local address 127.2.76.194:44373)
0809 23:14:05.564815 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.564817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.564823 (+ 6us) server_negotiation.cc:407] Connection header received
0809 23:14:05.564839 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.564840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.564846 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.564883 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:14:05.565015 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.565439 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.566466 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.566698 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.566761 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.566900 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:14:05.566925 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:14:05.566987 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.567034 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.567049 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 23:14:05.567153 5303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.564515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:44373 (local address 127.2.76.195:36999)
0809 23:14:05.564651 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:05.564659 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:05.564798 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:05.564893 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:05.564894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:05.564895 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:14:05.564938 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.564939 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.565660 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.565661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.566410 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:05.566413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.566761 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.566762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:05.566816 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.566829 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:14:05.566986 (+ 157us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:14:05.566988 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:14:05.567032 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 23:14:05.567066 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":81,"threads_started":1}
I20260809 23:14:05.573133 4294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:05.573201 4294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:05.573990 4294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da146264d5354636a20a4c850e8c709d: Sending DeleteTablet for 3 replicas of tablet 4ec72832c2ab4d869e63b2b899d2380c
I20260809 23:14:05.574221 4533 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ec72832c2ab4d869e63b2b899d2380c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:05 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:49568
I20260809 23:14:05.574298 4803 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ec72832c2ab4d869e63b2b899d2380c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:05 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:48780
I20260809 23:14:05.574389 5243 tablet_replica.cc:330] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9: stopping tablet replica
I20260809 23:14:05.574426 5250 tablet_replica.cc:330] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5: stopping tablet replica
I20260809 23:14:05.574445 5243 raft_consensus.cc:2229] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:05.574479 5243 raft_consensus.cc:2258] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:05.574482 5250 raft_consensus.cc:2229] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:14:05.574568 5250 raft_consensus.cc:2258] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:05.574638 5061 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ec72832c2ab4d869e63b2b899d2380c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:05 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:42248
I20260809 23:14:05.574710 5249 tablet_replica.cc:330] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f: stopping tablet replica
I20260809 23:14:05.574755 5249 raft_consensus.cc:2229] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:05.574795 5249 raft_consensus.cc:2258] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:05.575186 5250 ts_tablet_manager.cc:1905] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:05.575641 5243 ts_tablet_manager.cc:1905] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:05.576778 5250 ts_tablet_manager.cc:1918] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:14:05.576833 5250 log.cc:1196] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/4ec72832c2ab4d869e63b2b899d2380c
I20260809 23:14:05.577039 5250 ts_tablet_manager.cc:1939] T 4ec72832c2ab4d869e63b2b899d2380c P cd23852b8fb146bea7eba9caac34cdd5: Deleting consensus metadata
I20260809 23:14:05.577067 5243 ts_tablet_manager.cc:1918] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:14:05.577116 5243 log.cc:1196] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-0/wal/wals/4ec72832c2ab4d869e63b2b899d2380c
I20260809 23:14:05.577306 5243 ts_tablet_manager.cc:1939] T 4ec72832c2ab4d869e63b2b899d2380c P c201a739bce54c23b7add3aa5e9847f9: Deleting consensus metadata
I20260809 23:14:05.577458 4268 catalog_manager.cc:4841] TS cd23852b8fb146bea7eba9caac34cdd5 (127.2.76.194:44373): tablet 4ec72832c2ab4d869e63b2b899d2380c (table test-table [id=08f3579d4682491faa4820f5fa146af1]) successfully deleted
I20260809 23:14:05.577636 4271 catalog_manager.cc:4841] TS c201a739bce54c23b7add3aa5e9847f9 (127.2.76.193:36215): tablet 4ec72832c2ab4d869e63b2b899d2380c (table test-table [id=08f3579d4682491faa4820f5fa146af1]) successfully deleted
I20260809 23:14:05.577808 5249 ts_tablet_manager.cc:1905] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:05.579386 5249 ts_tablet_manager.cc:1918] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:14:05.579449 5249 log.cc:1196] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/4ec72832c2ab4d869e63b2b899d2380c
I20260809 23:14:05.579679 5249 ts_tablet_manager.cc:1939] T 4ec72832c2ab4d869e63b2b899d2380c P 34c4895b47e046d58b1f0bf95fc58f8f: Deleting consensus metadata
I20260809 23:14:05.579965 4266 catalog_manager.cc:4841] TS 34c4895b47e046d58b1f0bf95fc58f8f (127.2.76.195:46061): tablet 4ec72832c2ab4d869e63b2b899d2380c (table test-table [id=08f3579d4682491faa4820f5fa146af1]) successfully deleted
Aug 09 23:14:05 dist-test-slave-7vjv krb5kdc[4216](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317245, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.76.193@KRBTEST.COM
I20260809 23:14:05.594156 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.584347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49576 (local address 127.2.76.193:36215)
0809 23:14:05.584386 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.584388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.584494 (+ 106us) server_negotiation.cc:407] Connection header received
0809 23:14:05.584546 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.584547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.584556 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.584613 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:05.585307 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.586034 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.587051 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.587325 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.587387 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.590779 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:05.590787 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:05.590787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:05.590794 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:05.592762 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:05.593132 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:05.593135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:05.593135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:05.593194 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:05.593394 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:05.593395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:05.593396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:05.593550 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:05.593654 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.593978 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.594070 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 23:14:05.594911 4653 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49576
I20260809 23:14:05.602746 4375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:05.595833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52014 (local address 127.2.76.254:39573)
0809 23:14:05.595869 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:05.595871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:05.596069 (+ 198us) server_negotiation.cc:407] Connection header received
0809 23:14:05.596124 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:05.596125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:05.596134 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:05.596187 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:05.596704 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.597341 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.598254 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:05.598539 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:05.598603 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:05.599989 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:05.599990 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:05.599990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:05.599997 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:05.601589 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:05.601939 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:05.601940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:05.601941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:05.601995 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:05.602218 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:05.602219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:05.602219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:05.602343 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:05.602428 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:05.602554 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 23:14:05.602625 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 23:14:05.602977 4294 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52014
I20260809 23:14:05.603556 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 4415
I20260809 23:14:05.627691 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 4683
I20260809 23:14:05.638831 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 4937
I20260809 23:14:05.651865 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 4235
2026-08-09T23:14:05Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2419 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:14:05 dist-test-slave-7vjv krb5kdc[5320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:14:05 dist-test-slave-7vjv krb5kdc[5320](info): set up 2 sockets
Aug 09 23:14:05 dist-test-slave-7vjv krb5kdc[5320](info): commencing operation
krb5kdc: starting...
W20260809 23:14:06.796600 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:14:06 dist-test-slave-7vjv krb5kdc[5320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:14:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:14:06.904031 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:34155
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:44163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:34155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:06.976037 5338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:06.976296 5338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:06.976320 5338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:06.977576 5338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:06.977617 5338 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 23:14:06.977629 5338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:06.977643 5338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:06.977692 5338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:06.977710 5338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:14:06.979111 5338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:44163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:34155
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:34155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:06.979362 5338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:06.979585 5338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:06.984378 5357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:06.984381 5352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:06.984381 5353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:06.984382 5338 server_base.cc:1034] running on GCE node
I20260809 23:14:06.984876 5338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:06.985169 5338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:06.986303 5338 hybrid_clock.cc:648] HybridClock initialized: now 1786317246986293 us; error 26 us; skew 500 ppm
Aug 09 23:14:06 dist-test-slave-7vjv krb5kdc[5320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317246, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:06.993288 5338 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:14:06.993755 5338 webserver.cc:460] Webserver started at http://127.2.76.254:46177/ using document root <none> and password file <none>
I20260809 23:14:06.993963 5338 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:06.994005 5338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:06.994146 5338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:06.995100 5338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "3581bfe8026747c6885a2abb39e76255"
format_stamp: "Formatted at 2026-08-09 23:14:06 on dist-test-slave-7vjv"
server_key: "aa359d3029a03c753c0142fee25be82f"
server_key_iv: "125e232ce24100c54ea42065df627459"
server_key_version: "encryptionkey@0"
I20260809 23:14:06.995431 5338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "3581bfe8026747c6885a2abb39e76255"
format_stamp: "Formatted at 2026-08-09 23:14:06 on dist-test-slave-7vjv"
server_key: "aa359d3029a03c753c0142fee25be82f"
server_key_iv: "125e232ce24100c54ea42065df627459"
server_key_version: "encryptionkey@0"
I20260809 23:14:06.996727 5338 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:14:06.997452 5368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:06.997637 5338 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:06.997716 5338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "3581bfe8026747c6885a2abb39e76255"
format_stamp: "Formatted at 2026-08-09 23:14:06 on dist-test-slave-7vjv"
server_key: "aa359d3029a03c753c0142fee25be82f"
server_key_iv: "125e232ce24100c54ea42065df627459"
server_key_version: "encryptionkey@0"
I20260809 23:14:06.997763 5338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:07.033820 5338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:07.035396 5338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:07.035491 5338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:07.040045 5338 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:34155
I20260809 23:14:07.040143 5467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:34155 every 8 connection(s)
I20260809 23:14:07.040419 5338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:07.041002 5472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:07.042973 5472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255: Bootstrap starting.
I20260809 23:14:07.043494 5472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:07.043793 5472 log.cc:824] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:07.044484 5472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255: No bootstrap required, opened a new log
I20260809 23:14:07.046444 5472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3581bfe8026747c6885a2abb39e76255" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 34155 } }
I20260809 23:14:07.046591 5472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:07.046633 5472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3581bfe8026747c6885a2abb39e76255, State: Initialized, Role: FOLLOWER
I20260809 23:14:07.046828 5472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3581bfe8026747c6885a2abb39e76255" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 34155 } }
I20260809 23:14:07.046936 5472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:07.046981 5472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:07.047032 5472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:07.047540 5472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3581bfe8026747c6885a2abb39e76255" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 34155 } }
I20260809 23:14:07.047659 5472 leader_election.cc:302] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3581bfe8026747c6885a2abb39e76255; no voters:
I20260809 23:14:07.047848 5472 leader_election.cc:288] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:07.047895 5476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:07.048087 5476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [term 1 LEADER]: Becoming Leader. State: Replica: 3581bfe8026747c6885a2abb39e76255, State: Running, Role: LEADER
I20260809 23:14:07.048144 5472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:07.048219 5476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3581bfe8026747c6885a2abb39e76255" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 34155 } }
I20260809 23:14:07.048499 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 5338
I20260809 23:14:07.048604 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:07.048645 5479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3581bfe8026747c6885a2abb39e76255. Latest consensus state: current_term: 1 leader_uuid: "3581bfe8026747c6885a2abb39e76255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3581bfe8026747c6885a2abb39e76255" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 34155 } } }
I20260809 23:14:07.048714 5479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:07.048780 2355 external_mini_cluster.cc:1428] Setting key 801fb71a038a165f162b68d4c871c205
I20260809 23:14:07.048864 5477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3581bfe8026747c6885a2abb39e76255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3581bfe8026747c6885a2abb39e76255" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 34155 } } }
I20260809 23:14:07.049189 5486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:07.048914 5477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:07.049832 5486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:07.052733 5486 catalog_manager.cc:1347] Generated new cluster ID: 7112a9e9fbe249b0b9579131f31ebd9f
I20260809 23:14:07.052783 5486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:14:07 dist-test-slave-7vjv krb5kdc[5320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:07.060194 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.049184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58554 (local address 127.2.76.254:34155)
0809 23:14:07.049383 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.049392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.049757 (+ 365us) server_negotiation.cc:407] Connection header received
0809 23:14:07.049862 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.049868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.049922 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.050048 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:07.050869 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.052073 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.053001 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.053362 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.053439 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.055997 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:07.056001 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:07.056001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:07.056012 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:07.058492 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.058879 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.058882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.058882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.058998 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.059194 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.059195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.059195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.059478 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:07.059636 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.059756 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.059965 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":78,"threads_started":1}
I20260809 23:14:07.071841 5486 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:07.072446 5486 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:07.081014 5486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255: Generated new TSK 0
I20260809 23:14:07.081180 5486 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:14:07.113201 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:34155
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:44163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:07.187207 5520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:07.187363 5520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:07.187383 5520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:07.188369 5520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:07.188403 5520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:07.188421 5520 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 23:14:07.188432 5520 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:14:07.188447 5520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:14:07.189689 5520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:44163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.76.254:34155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:07.189853 5520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:07.190003 5520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:07.194460 5535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:07.194465 5532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:07.194586 5533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:07.194824 5520 server_base.cc:1034] running on GCE node
I20260809 23:14:07.195034 5520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:07.195276 5520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:07.196441 5520 hybrid_clock.cc:648] HybridClock initialized: now 1786317247196435 us; error 35 us; skew 500 ppm
Aug 09 23:14:07 dist-test-slave-7vjv krb5kdc[5320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317247, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:07.198915 5520 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:14:07.199287 5520 webserver.cc:460] Webserver started at http://127.2.76.193:35737/ using document root <none> and password file <none>
I20260809 23:14:07.199467 5520 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:07.199504 5520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:07.199625 5520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:07.200639 5520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f"
format_stamp: "Formatted at 2026-08-09 23:14:07 on dist-test-slave-7vjv"
server_key: "7e86b7b348ca7f5bf2932fc35942170e"
server_key_iv: "e441696768145b8da21d7b14ae333f8e"
server_key_version: "encryptionkey@0"
I20260809 23:14:07.200994 5520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f"
format_stamp: "Formatted at 2026-08-09 23:14:07 on dist-test-slave-7vjv"
server_key: "7e86b7b348ca7f5bf2932fc35942170e"
server_key_iv: "e441696768145b8da21d7b14ae333f8e"
server_key_version: "encryptionkey@0"
I20260809 23:14:07.202330 5520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:14:07.202919 5544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:07.203070 5520 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:14:07.203140 5520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f"
format_stamp: "Formatted at 2026-08-09 23:14:07 on dist-test-slave-7vjv"
server_key: "7e86b7b348ca7f5bf2932fc35942170e"
server_key_iv: "e441696768145b8da21d7b14ae333f8e"
server_key_version: "encryptionkey@0"
I20260809 23:14:07.203186 5520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:07.211040 5520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:07.213136 5520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:07.213243 5520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:07.213651 5520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:07.213692 5520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:07.213716 5520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:07.213768 5520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:14:07 dist-test-slave-7vjv krb5kdc[5320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317247, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:07.220717 5520 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:42953
I20260809 23:14:07.220799 5728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:42953 every 8 connection(s)
I20260809 23:14:07.221138 5520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:14:07.223557 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.214081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:51491 (local address 127.2.76.254:34155)
0809 23:14:07.214152 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.214155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.214314 (+ 159us) server_negotiation.cc:407] Connection header received
0809 23:14:07.214667 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.214668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.214683 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.214768 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:07.215350 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.216054 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.217457 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.217773 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.217869 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.221050 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:07.221051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:07.221052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:07.221059 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:07.222397 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.222742 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.222743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.222743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.222796 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.223060 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.223061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.223062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.223169 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:07.223231 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.223437 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.223501 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:14:07.223664 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.214059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:34155 (local address 127.2.76.193:51491)
0809 23:14:07.214197 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:07.214230 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:07.214579 (+ 349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:07.214804 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:07.214807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:07.215016 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:07.215249 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.215255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.216181 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.216182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.217357 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.217361 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.217887 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.217888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.218050 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.218583 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:07.218594 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:07.220953 (+ 2359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:07.222468 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:07.222474 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:07.222476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:07.222675 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:07.222879 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:07.222880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:07.222880 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:07.222967 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:07.223231 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:07.223235 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:07.223339 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 23:14:07.223407 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 23:14:07.223515 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
I20260809 23:14:07.224171 5729 heartbeater.cc:347] Connected to a master server at 127.2.76.254:34155
I20260809 23:14:07.224247 5729 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:07.224406 5729 heartbeater.cc:511] Master 127.2.76.254:34155 requested a full tablet report, sending...
I20260809 23:14:07.224898 5391 ts_manager.cc:194] Registered new tserver with Master: 4cb4fb59fdad44e4a5f069672f4a7b9f (127.2.76.193:42953)
I20260809 23:14:07.225533 5391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:51491
I20260809 23:14:07.227192 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 5520
I20260809 23:14:07.227301 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:14:07.227487 2355 external_mini_cluster.cc:1428] Setting key 54ac9d9962e05571d8b905e973683d24
I20260809 23:14:07.230346 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.225131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:42535 (local address 127.2.76.254:34155)
0809 23:14:07.225700 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.225702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.225709 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:14:07.225726 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.225726 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.225735 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.225779 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:07.226094 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.226466 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.227051 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.227203 (+ 152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.227248 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.228173 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:07.228174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:07.228174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:07.228179 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:07.229362 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.229680 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.229681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.229681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.229733 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.229913 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.229914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.229915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.230009 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:07.230074 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.230209 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.230275 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552}
I20260809 23:14:07.230355 5739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.225099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:34155 (local address 127.2.76.193:42535)
0809 23:14:07.225228 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:07.225236 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:07.225274 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:07.225787 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:07.225789 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:07.225966 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:07.226024 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.226025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.226550 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.226550 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.226986 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.226988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.227279 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.227280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.227364 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.227775 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:07.227780 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:07.228082 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:07.229453 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:07.229454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:07.229454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:07.229611 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:07.229801 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:07.229801 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:07.229802 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:07.229859 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:07.230068 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:07.230069 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:07.230109 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 23:14:07.230187 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 23:14:07.230275 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:14:07.251217 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:34155
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:44163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:07.323011 5742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:07.323161 5742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:07.323182 5742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:07.324226 5742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:07.324263 5742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:07.324282 5742 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 23:14:07.324293 5742 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:14:07.324308 5742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:14:07.325685 5742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:44163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.76.254:34155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:07.325845 5742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:07.325992 5742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:07.330521 5749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:07.330541 5751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:07.330655 5742 server_base.cc:1034] running on GCE node
W20260809 23:14:07.330521 5753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:07.330893 5742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:07.331147 5742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:07.332295 5742 hybrid_clock.cc:648] HybridClock initialized: now 1786317247332287 us; error 24 us; skew 500 ppm
Aug 09 23:14:07 dist-test-slave-7vjv krb5kdc[5320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317247, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:07.334595 5742 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:14:07.335032 5742 webserver.cc:460] Webserver started at http://127.2.76.194:34165/ using document root <none> and password file <none>
I20260809 23:14:07.335263 5742 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:07.335304 5742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:07.335427 5742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:07.336341 5742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "1abdb315effc4e49a17e89efb29da6e0"
format_stamp: "Formatted at 2026-08-09 23:14:07 on dist-test-slave-7vjv"
server_key: "fe83a1909fa9ef855494c925a86c66ac"
server_key_iv: "e69c6d65466585385005ab1faf208ed6"
server_key_version: "encryptionkey@0"
I20260809 23:14:07.336651 5742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "1abdb315effc4e49a17e89efb29da6e0"
format_stamp: "Formatted at 2026-08-09 23:14:07 on dist-test-slave-7vjv"
server_key: "fe83a1909fa9ef855494c925a86c66ac"
server_key_iv: "e69c6d65466585385005ab1faf208ed6"
server_key_version: "encryptionkey@0"
I20260809 23:14:07.337774 5742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:07.338382 5769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:07.338541 5742 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:07.338629 5742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "1abdb315effc4e49a17e89efb29da6e0"
format_stamp: "Formatted at 2026-08-09 23:14:07 on dist-test-slave-7vjv"
server_key: "fe83a1909fa9ef855494c925a86c66ac"
server_key_iv: "e69c6d65466585385005ab1faf208ed6"
server_key_version: "encryptionkey@0"
I20260809 23:14:07.338706 5742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:07.344714 5742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:07.346276 5742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:07.346410 5742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:07.346803 5742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:07.346860 5742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:07.346900 5742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:07.346928 5742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:14:07 dist-test-slave-7vjv krb5kdc[5320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317247, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:07.354053 5742 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:33169
I20260809 23:14:07.354107 5962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:33169 every 8 connection(s)
I20260809 23:14:07.354389 5742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:14:07.355510 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 5742
I20260809 23:14:07.355602 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:14:07.355777 2355 external_mini_cluster.cc:1428] Setting key d4a98bbab583c5af7ebee30f82464c86
I20260809 23:14:07.356688 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.347394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:49191 (local address 127.2.76.254:34155)
0809 23:14:07.347464 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.347466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.347675 (+ 209us) server_negotiation.cc:407] Connection header received
0809 23:14:07.348358 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.348359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.348384 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.348440 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:07.349222 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.349960 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.351427 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.351705 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.351765 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.353973 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:07.353975 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:07.353975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:07.353982 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:07.355298 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.355762 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.355763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.355763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.355840 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.356109 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.356110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.356110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.356226 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:07.356277 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.356570 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.356631 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:14:07.356937 5787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.347345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:34155 (local address 127.2.76.194:49191)
0809 23:14:07.347538 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:07.347584 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:07.348229 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:07.348524 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:07.348529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:07.348867 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:07.349137 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.349144 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.350077 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.350078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.351328 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.351332 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.351802 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.351804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.351961 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.352486 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:07.352496 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:07.353886 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:07.355396 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:07.355401 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:07.355403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:07.355696 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:07.355913 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:07.355914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:07.355914 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:07.356023 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:07.356282 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:07.356287 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:07.356437 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 23:14:07.356553 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 23:14:07.356721 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
I20260809 23:14:07.357366 5963 heartbeater.cc:347] Connected to a master server at 127.2.76.254:34155
I20260809 23:14:07.357478 5963 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:07.357666 5963 heartbeater.cc:511] Master 127.2.76.254:34155 requested a full tablet report, sending...
I20260809 23:14:07.358204 5393 ts_manager.cc:194] Registered new tserver with Master: 1abdb315effc4e49a17e89efb29da6e0 (127.2.76.194:33169)
I20260809 23:14:07.358718 5393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:49191
I20260809 23:14:07.364405 5973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.358296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:34155 (local address 127.2.76.194:47301)
0809 23:14:07.358435 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:07.358444 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:07.358487 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:07.359107 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:07.359108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:07.359283 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:07.359331 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.359332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.360199 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.360199 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.360628 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.360629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.361074 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.361074 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.361123 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.361537 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:07.361541 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:07.361771 (+ 230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:07.363465 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:07.363465 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:07.363466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:07.363620 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:07.363811 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:07.363812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:07.363812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:07.363858 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:07.364200 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:07.364201 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:07.364235 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 23:14:07.364280 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 23:14:07.364345 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260809 23:14:07.364482 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.358923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:47301 (local address 127.2.76.254:34155)
0809 23:14:07.358983 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.358985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.358994 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:14:07.359014 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.359015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.359024 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.359089 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:07.359413 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.360103 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.360703 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.360992 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.361075 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.361872 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:07.361873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:07.361874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:07.361881 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:07.363380 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.363685 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.363686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.363686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.363745 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.363915 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.363916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.363917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.364126 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:07.364195 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.364325 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.364396 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:14:07.386960 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:34155
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:44163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:07.461042 5975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:07.461201 5975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:07.461241 5975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:07.462324 5975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:07.462365 5975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:07.462409 5975 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 23:14:07.462438 5975 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:14:07.462467 5975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:14:07.463768 5975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:44163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.76.254:34155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:07.463973 5975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:07.464246 5975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:07.468701 5988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:07.468868 5975 server_base.cc:1034] running on GCE node
W20260809 23:14:07.468714 5987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:07.468786 5991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:07.469111 5975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:07.469369 5975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:07.470530 5975 hybrid_clock.cc:648] HybridClock initialized: now 1786317247470516 us; error 37 us; skew 500 ppm
Aug 09 23:14:07 dist-test-slave-7vjv krb5kdc[5320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317247, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:07.472700 5975 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:14:07.473119 5975 webserver.cc:460] Webserver started at http://127.2.76.195:37497/ using document root <none> and password file <none>
I20260809 23:14:07.473347 5975 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:07.473414 5975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:07.473568 5975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:07.474521 5975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "0967cb4f28f24d14afa07e833eda7399"
format_stamp: "Formatted at 2026-08-09 23:14:07 on dist-test-slave-7vjv"
server_key: "807357ee73d814dc7ef18b63c6253974"
server_key_iv: "dc5fe83f9e401370aecc37c04a4a8bc4"
server_key_version: "encryptionkey@0"
I20260809 23:14:07.474852 5975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "0967cb4f28f24d14afa07e833eda7399"
format_stamp: "Formatted at 2026-08-09 23:14:07 on dist-test-slave-7vjv"
server_key: "807357ee73d814dc7ef18b63c6253974"
server_key_iv: "dc5fe83f9e401370aecc37c04a4a8bc4"
server_key_version: "encryptionkey@0"
I20260809 23:14:07.476126 5975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:14:07.476811 6005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:07.476974 5975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:07.477066 5975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "0967cb4f28f24d14afa07e833eda7399"
format_stamp: "Formatted at 2026-08-09 23:14:07 on dist-test-slave-7vjv"
server_key: "807357ee73d814dc7ef18b63c6253974"
server_key_iv: "dc5fe83f9e401370aecc37c04a4a8bc4"
server_key_version: "encryptionkey@0"
I20260809 23:14:07.477146 5975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:07.494320 5975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:07.495790 5975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:07.495927 5975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:07.496511 5975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:07.496565 5975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:07.496595 5975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:07.496649 5975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:14:07 dist-test-slave-7vjv krb5kdc[5320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317247, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:07.505158 5975 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:37023
I20260809 23:14:07.505215 6202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:37023 every 8 connection(s)
I20260809 23:14:07.505535 5975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:14:07.506054 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.496921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:44927 (local address 127.2.76.254:34155)
0809 23:14:07.496961 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.496963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.497163 (+ 200us) server_negotiation.cc:407] Connection header received
0809 23:14:07.497512 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.497513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.497528 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.497601 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:07.498072 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.498817 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.499790 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.499965 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.500043 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.503064 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:07.503066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:07.503066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:07.503073 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:07.504519 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.504968 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.504970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.504970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.505029 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.505287 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.505288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.505288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.505427 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:07.505508 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.505816 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.505934 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:14:07.506165 6019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.496918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:34155 (local address 127.2.76.195:44927)
0809 23:14:07.497063 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:07.497091 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:07.497440 (+ 349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:07.497640 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:07.497642 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:07.497836 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:07.497985 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.497988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.498910 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.498912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.499719 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.499722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.500063 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.500065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.500211 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.500781 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:07.500802 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:07.502972 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:07.504648 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:07.504655 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:07.504658 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:07.504881 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:07.505099 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:07.505099 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:07.505100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:07.505223 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:07.505502 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:07.505507 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:07.505661 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 23:14:07.505755 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 23:14:07.505935 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260809 23:14:07.506764 6204 heartbeater.cc:347] Connected to a master server at 127.2.76.254:34155
I20260809 23:14:07.506856 6204 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:07.507014 6204 heartbeater.cc:511] Master 127.2.76.254:34155 requested a full tablet report, sending...
I20260809 23:14:07.507368 5391 ts_manager.cc:194] Registered new tserver with Master: 0967cb4f28f24d14afa07e833eda7399 (127.2.76.195:37023)
I20260809 23:14:07.507848 5391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:44927
I20260809 23:14:07.511181 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 5975
I20260809 23:14:07.511287 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:14:07.511453 2355 external_mini_cluster.cc:1428] Setting key aa597dc459f23ef654dba149ec0f135e
I20260809 23:14:07.512442 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:14:07.513069 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.507794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:41889 (local address 127.2.76.254:34155)
0809 23:14:07.507837 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.507838 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.507898 (+ 60us) server_negotiation.cc:407] Connection header received
0809 23:14:07.507963 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.507963 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.507972 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.508040 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:07.508346 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.508764 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.509395 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.509571 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.509665 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.510492 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:07.510494 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:07.510494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:07.510500 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:07.511870 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.512246 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.512247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.512248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.512422 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.512585 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.512586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.512586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.512685 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:07.512751 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.512917 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.512985 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:14:07.513098 6214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.507710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:34155 (local address 127.2.76.195:41889)
0809 23:14:07.507847 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:07.507857 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:07.507912 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:07.508045 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:07.508045 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:07.508224 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:07.508270 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.508271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.508855 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.508856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.509312 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.509314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.509665 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.509666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.509734 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.510153 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:07.510157 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:07.510391 (+ 234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:07.511947 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:07.511948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:07.511948 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:07.512173 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:07.512479 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:07.512480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:07.512481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:07.512540 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:07.512756 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:07.512757 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:07.512797 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 23:14:07.512875 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 23:14:07.512987 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
Aug 09 23:14:07 dist-test-slave-7vjv krb5kdc[5320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 23:14:07 dist-test-slave-7vjv krb5kdc[5320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:07.534888 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.529249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58556 (local address 127.2.76.254:34155)
0809 23:14:07.529291 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.529292 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.529299 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:14:07.529322 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.529322 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.529329 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.529372 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:07.529824 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.530186 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.530790 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.530939 (+ 149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.530981 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.532854 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:07.532856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:07.532856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:07.532862 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:07.534181 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.534428 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.534429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.534429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.534470 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.534602 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.534602 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.534602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.534670 (+ 68us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:07.534727 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.534797 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.534843 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 23:14:07.536128 5391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:14:07.537490 5391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:14:07.543059 6232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.539194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:37023 (local address 127.0.0.1:34992)
0809 23:14:07.539490 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:07.539522 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:07.539597 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:07.539840 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:07.539844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:07.539846 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:07.539917 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.539918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.541103 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.541104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.542241 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.542243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.542799 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.542800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.542870 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.542881 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 23:14:07.542939 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 23:14:07.542981 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":34,"thread_start_us":47,"threads_started":1}
I20260809 23:14:07.543282 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.539279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60514 (local address 127.2.76.193:42953)
0809 23:14:07.539517 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.539522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.539676 (+ 154us) server_negotiation.cc:407] Connection header received
0809 23:14:07.539715 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.539717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.539731 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.539796 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:07.540016 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.541071 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.542197 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.542841 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.542922 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.542975 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.543030 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.543068 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":94,"threads_started":1}
I20260809 23:14:07.543301 6227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.539254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34992 (local address 127.2.76.195:37023)
0809 23:14:07.539363 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.539367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.539675 (+ 308us) server_negotiation.cc:407] Connection header received
0809 23:14:07.539712 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.539713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.539726 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.539796 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:07.540006 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.541022 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.542316 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.542718 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.542796 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.542843 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.543207 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.543250 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":59,"threads_started":1}
I20260809 23:14:07.543133 6225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.539177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:42953 (local address 127.0.0.1:60514)
0809 23:14:07.539484 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:07.539522 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:07.539597 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:07.539840 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:07.539844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:07.539848 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:07.539915 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.539917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.541141 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.541142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.542118 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.542120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.542921 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.542922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.542970 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.542972 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:14:07.543016 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 23:14:07.543043 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":107,"thread_start_us":49,"threads_started":1}
I20260809 23:14:07.543442 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.539465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50588 (local address 127.2.76.194:33169)
0809 23:14:07.539607 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.539611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.539975 (+ 364us) server_negotiation.cc:407] Connection header received
0809 23:14:07.540136 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.540139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.540155 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.540221 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:07.540415 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.541141 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.542303 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.542897 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.542948 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.542990 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.543302 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.543349 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260809 23:14:07.543236 6226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.539390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:33169 (local address 127.0.0.1:50588)
0809 23:14:07.539887 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:07.539896 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:07.539971 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:07.540296 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:07.540297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:07.540300 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:07.540338 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.540339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.541202 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.541203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.542223 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.542226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.543032 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.543034 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.543084 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.543089 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:14:07.543134 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 23:14:07.543170 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":6,"thread_start_us":63,"threads_started":1}
I20260809 23:14:07.543975 6080 tablet_service.cc:1467] Processing CreateTablet for tablet fa771cb02c2149829ea8631878bac4fa (DEFAULT_TABLE table=test-table [id=f901749fb4dd495ab234fe49e17880f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:07.544041 5628 tablet_service.cc:1467] Processing CreateTablet for tablet fa771cb02c2149829ea8631878bac4fa (DEFAULT_TABLE table=test-table [id=f901749fb4dd495ab234fe49e17880f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:07.544341 6080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa771cb02c2149829ea8631878bac4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:07.544271 5850 tablet_service.cc:1467] Processing CreateTablet for tablet fa771cb02c2149829ea8631878bac4fa (DEFAULT_TABLE table=test-table [id=f901749fb4dd495ab234fe49e17880f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:07.544368 5628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa771cb02c2149829ea8631878bac4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:07.544552 5850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa771cb02c2149829ea8631878bac4fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:07.546571 6237 tablet_bootstrap.cc:492] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399: Bootstrap starting.
I20260809 23:14:07.547365 6237 tablet_bootstrap.cc:654] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:07.547673 6237 log.cc:824] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:07.547693 6238 tablet_bootstrap.cc:492] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f: Bootstrap starting.
I20260809 23:14:07.547943 6239 tablet_bootstrap.cc:492] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0: Bootstrap starting.
I20260809 23:14:07.548297 6238 tablet_bootstrap.cc:654] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:07.548566 6237 tablet_bootstrap.cc:492] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399: No bootstrap required, opened a new log
I20260809 23:14:07.548585 6239 tablet_bootstrap.cc:654] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:07.548633 6237 ts_tablet_manager.cc:1397] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:07.548666 6238 log.cc:824] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:07.548907 6239 log.cc:824] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:07.549742 6238 tablet_bootstrap.cc:492] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f: No bootstrap required, opened a new log
I20260809 23:14:07.549753 6239 tablet_bootstrap.cc:492] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0: No bootstrap required, opened a new log
I20260809 23:14:07.549788 6238 ts_tablet_manager.cc:1397] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:07.549803 6239 ts_tablet_manager.cc:1397] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:07.550565 6237 raft_consensus.cc:348] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } }
I20260809 23:14:07.550709 6237 raft_consensus.cc:374] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:07.550740 6237 raft_consensus.cc:729] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0967cb4f28f24d14afa07e833eda7399, State: Initialized, Role: FOLLOWER
I20260809 23:14:07.550840 6237 consensus_queue.cc:260] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } }
I20260809 23:14:07.551077 6237 ts_tablet_manager.cc:1428] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:07.551136 6204 heartbeater.cc:503] Master 127.2.76.254:34155 was elected leader, sending a full tablet report...
I20260809 23:14:07.551100 6238 raft_consensus.cc:348] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } }
I20260809 23:14:07.551527 6238 raft_consensus.cc:374] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:07.551599 6238 raft_consensus.cc:729] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb4fb59fdad44e4a5f069672f4a7b9f, State: Initialized, Role: FOLLOWER
I20260809 23:14:07.551713 6238 consensus_queue.cc:260] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } }
I20260809 23:14:07.551821 6239 raft_consensus.cc:348] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } }
I20260809 23:14:07.551990 5729 heartbeater.cc:503] Master 127.2.76.254:34155 was elected leader, sending a full tablet report...
I20260809 23:14:07.551992 6239 raft_consensus.cc:374] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:07.552038 6239 raft_consensus.cc:729] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1abdb315effc4e49a17e89efb29da6e0, State: Initialized, Role: FOLLOWER
I20260809 23:14:07.552062 6238 ts_tablet_manager.cc:1428] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:07.552150 6239 consensus_queue.cc:260] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } }
I20260809 23:14:07.552371 6239 ts_tablet_manager.cc:1428] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 23:14:07.552495 5963 heartbeater.cc:503] Master 127.2.76.254:34155 was elected leader, sending a full tablet report...
I20260809 23:14:07.603260 6249 raft_consensus.cc:482] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:14:07.603359 6249 raft_consensus.cc:504] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } }
I20260809 23:14:07.603582 6249 leader_election.cc:288] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0967cb4f28f24d14afa07e833eda7399 (127.2.76.195:37023), 1abdb315effc4e49a17e89efb29da6e0 (127.2.76.194:33169)
W20260809 23:14:07.605170 5965 tablet.cc:2369] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:14:07.606209 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.603823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:35747 (local address 127.2.76.194:33169)
0809 23:14:07.603863 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.603864 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.603872 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:07.603894 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.603895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.603905 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.603969 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:07.604147 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.604608 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.605535 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.605927 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.605987 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.605999 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.606126 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.606154 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:14:07.606230 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.603755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:33169 (local address 127.2.76.193:35747)
0809 23:14:07.603791 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:07.603801 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:07.603836 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:07.603996 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:07.603997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:07.603999 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:07.604057 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.604058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.604680 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.604681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.605460 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.605462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.605995 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.605996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.606060 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.606063 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:14:07.606119 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 23:14:07.606152 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":18}
I20260809 23:14:07.606496 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa771cb02c2149829ea8631878bac4fa" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1abdb315effc4e49a17e89efb29da6e0" is_pre_election: true
I20260809 23:14:07.606544 6227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.603818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:47733 (local address 127.2.76.195:37023)
0809 23:14:07.603857 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.603858 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.604008 (+ 150us) server_negotiation.cc:407] Connection header received
0809 23:14:07.604045 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.604046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.604055 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.604144 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:07.604273 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.604656 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.605721 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.606166 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.606219 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.606225 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.606463 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.606494 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:14:07.606556 6251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.603755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:37023 (local address 127.2.76.193:47733)
0809 23:14:07.603934 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:07.603944 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:07.604004 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:07.604152 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:07.604153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:07.604154 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:07.604214 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.604215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.604735 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.604735 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.605646 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.605648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.606257 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.606258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.606372 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.606375 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:14:07.606431 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 23:14:07.606468 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":40,"thread_start_us":86,"threads_started":1}
I20260809 23:14:07.606640 5880 raft_consensus.cc:2454] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 0.
I20260809 23:14:07.606817 5548 leader_election.cc:302] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1abdb315effc4e49a17e89efb29da6e0, 4cb4fb59fdad44e4a5f069672f4a7b9f; no voters:
I20260809 23:14:07.606868 6119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa771cb02c2149829ea8631878bac4fa" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0967cb4f28f24d14afa07e833eda7399" is_pre_election: true
I20260809 23:14:07.606943 6249 raft_consensus.cc:2781] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:14:07.606994 6249 raft_consensus.cc:482] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:14:07.607002 6119 raft_consensus.cc:2454] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 0.
I20260809 23:14:07.607019 6249 raft_consensus.cc:3037] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:07.607892 6249 raft_consensus.cc:504] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } }
I20260809 23:14:07.608028 6249 leader_election.cc:288] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 election: Requested vote from peers 0967cb4f28f24d14afa07e833eda7399 (127.2.76.195:37023), 1abdb315effc4e49a17e89efb29da6e0 (127.2.76.194:33169)
I20260809 23:14:07.608171 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa771cb02c2149829ea8631878bac4fa" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1abdb315effc4e49a17e89efb29da6e0"
I20260809 23:14:07.608242 5880 raft_consensus.cc:3037] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:07.608796 5880 raft_consensus.cc:2454] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 1.
I20260809 23:14:07.608934 6119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa771cb02c2149829ea8631878bac4fa" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0967cb4f28f24d14afa07e833eda7399"
I20260809 23:14:07.608986 5548 leader_election.cc:302] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1abdb315effc4e49a17e89efb29da6e0, 4cb4fb59fdad44e4a5f069672f4a7b9f; no voters:
I20260809 23:14:07.609006 6119 raft_consensus.cc:3037] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:07.609069 6249 raft_consensus.cc:2781] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:07.609194 6249 raft_consensus.cc:686] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 LEADER]: Becoming Leader. State: Replica: 4cb4fb59fdad44e4a5f069672f4a7b9f, State: Running, Role: LEADER
I20260809 23:14:07.609267 6249 consensus_queue.cc:237] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } }
I20260809 23:14:07.609782 6119 raft_consensus.cc:2454] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 1.
I20260809 23:14:07.609961 5390 catalog_manager.cc:5495] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cb4fb59fdad44e4a5f069672f4a7b9f (127.2.76.193). New cstate: current_term: 1 leader_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } health_report { overall_health: UNKNOWN } } }
Aug 09 23:14:07 dist-test-slave-7vjv krb5kdc[5320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317246, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:07.642408 6266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.636546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:34155 (local address 127.0.0.1:58558)
0809 23:14:07.636696 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:07.636704 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:07.636753 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:07.636851 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:07.636852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:07.637085 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:07.637120 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.637121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.637616 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.637616 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.638038 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.638039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.638295 (+ 256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.638296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.638338 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.638714 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:07.638718 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:07.639865 (+ 1147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:07.641464 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:07.641471 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:07.641473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:07.641640 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:07.641822 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:07.641822 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:07.641823 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:07.641876 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:07.642086 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:07.642093 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:07.642155 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 23:14:07.642208 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 23:14:07.642305 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260809 23:14:07.642423 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.636593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58558 (local address 127.2.76.254:34155)
0809 23:14:07.636655 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.636657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.636757 (+ 100us) server_negotiation.cc:407] Connection header received
0809 23:14:07.636787 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.636788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.636794 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.636835 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:07.637178 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.637557 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.638098 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.638240 (+ 142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.638280 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.639948 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:07.639949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:07.639950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:07.639956 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:07.641382 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.641706 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.641707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.641707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.641759 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:07.641934 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:07.641935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:07.641935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:07.642026 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:07.642094 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.642238 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.642349 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":38}
I20260809 23:14:07.644063 5390 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:58558:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 23:14:07.644280 5390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:14:07.645423 5628 tablet_service.cc:1467] Processing CreateTablet for tablet e180bc8a520c49f6a0f48235f43778af (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=078b8a320a894b85af017d142a311d7d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:14:07.645486 6080 tablet_service.cc:1467] Processing CreateTablet for tablet e180bc8a520c49f6a0f48235f43778af (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=078b8a320a894b85af017d142a311d7d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:14:07.645501 5850 tablet_service.cc:1467] Processing CreateTablet for tablet e180bc8a520c49f6a0f48235f43778af (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=078b8a320a894b85af017d142a311d7d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:14:07.645606 6080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e180bc8a520c49f6a0f48235f43778af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:07.645612 5850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e180bc8a520c49f6a0f48235f43778af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:07.645613 5628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e180bc8a520c49f6a0f48235f43778af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:07.646589 6237 tablet_bootstrap.cc:492] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399: Bootstrap starting.
I20260809 23:14:07.646919 6239 tablet_bootstrap.cc:492] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0: Bootstrap starting.
I20260809 23:14:07.646929 6238 tablet_bootstrap.cc:492] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f: Bootstrap starting.
I20260809 23:14:07.647207 6237 tablet_bootstrap.cc:654] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:07.647362 6238 tablet_bootstrap.cc:654] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:07.647457 6239 tablet_bootstrap.cc:654] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:07.647939 6237 tablet_bootstrap.cc:492] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399: No bootstrap required, opened a new log
I20260809 23:14:07.648008 6237 ts_tablet_manager.cc:1397] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:07.648195 6237 raft_consensus.cc:348] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.648262 6237 raft_consensus.cc:374] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:07.648283 6237 raft_consensus.cc:729] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0967cb4f28f24d14afa07e833eda7399, State: Initialized, Role: FOLLOWER
I20260809 23:14:07.648310 6239 tablet_bootstrap.cc:492] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0: No bootstrap required, opened a new log
I20260809 23:14:07.648327 6237 consensus_queue.cc:260] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.648360 6239 ts_tablet_manager.cc:1397] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 23:14:07.648422 6237 ts_tablet_manager.cc:1428] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:07.648473 6248 tablet_replica.cc:439] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } }
I20260809 23:14:07.648496 6239 raft_consensus.cc:348] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.648563 6239 raft_consensus.cc:374] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:07.648567 6248 tablet_replica.cc:442] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:14:07.648586 6239 raft_consensus.cc:729] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1abdb315effc4e49a17e89efb29da6e0, State: Initialized, Role: FOLLOWER
I20260809 23:14:07.648622 6239 consensus_queue.cc:260] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.648705 6239 ts_tablet_manager.cc:1428] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:07.648743 6238 tablet_bootstrap.cc:492] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f: No bootstrap required, opened a new log
I20260809 23:14:07.648785 6238 ts_tablet_manager.cc:1397] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:14:07.648895 6250 tablet_replica.cc:439] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } }
I20260809 23:14:07.648916 6238 raft_consensus.cc:348] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.648967 6250 tablet_replica.cc:442] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:14:07.648985 6238 raft_consensus.cc:374] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:07.649005 6238 raft_consensus.cc:729] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb4fb59fdad44e4a5f069672f4a7b9f, State: Initialized, Role: FOLLOWER
I20260809 23:14:07.649044 6238 consensus_queue.cc:260] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.649155 6238 ts_tablet_manager.cc:1428] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:07.649163 6249 tablet_replica.cc:439] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } }
I20260809 23:14:07.649262 6249 tablet_replica.cc:442] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:14:07.653024 6249 raft_consensus.cc:482] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:14:07.653136 6249 raft_consensus.cc:504] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.653299 6249 leader_election.cc:288] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1abdb315effc4e49a17e89efb29da6e0 (127.2.76.194:33169), 0967cb4f28f24d14afa07e833eda7399 (127.2.76.195:37023)
I20260809 23:14:07.653405 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e180bc8a520c49f6a0f48235f43778af" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1abdb315effc4e49a17e89efb29da6e0" is_pre_election: true
I20260809 23:14:07.653424 6119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e180bc8a520c49f6a0f48235f43778af" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0967cb4f28f24d14afa07e833eda7399" is_pre_election: true
I20260809 23:14:07.653491 5880 raft_consensus.cc:2454] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 0.
I20260809 23:14:07.653508 6119 raft_consensus.cc:2454] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 0.
I20260809 23:14:07.653606 5548 leader_election.cc:302] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1abdb315effc4e49a17e89efb29da6e0, 4cb4fb59fdad44e4a5f069672f4a7b9f; no voters:
I20260809 23:14:07.653682 6249 raft_consensus.cc:2781] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:14:07.653712 6249 raft_consensus.cc:482] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:14:07.653729 6249 raft_consensus.cc:3037] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:07.654173 6249 raft_consensus.cc:504] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.654310 6249 leader_election.cc:288] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 election: Requested vote from peers 1abdb315effc4e49a17e89efb29da6e0 (127.2.76.194:33169), 0967cb4f28f24d14afa07e833eda7399 (127.2.76.195:37023)
I20260809 23:14:07.654547 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e180bc8a520c49f6a0f48235f43778af" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1abdb315effc4e49a17e89efb29da6e0"
I20260809 23:14:07.654541 6119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e180bc8a520c49f6a0f48235f43778af" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0967cb4f28f24d14afa07e833eda7399"
I20260809 23:14:07.654608 5880 raft_consensus.cc:3037] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:07.654608 6119 raft_consensus.cc:3037] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:07.655236 6119 raft_consensus.cc:2454] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 1.
I20260809 23:14:07.655236 5880 raft_consensus.cc:2454] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 1.
I20260809 23:14:07.655431 5545 leader_election.cc:302] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0967cb4f28f24d14afa07e833eda7399, 4cb4fb59fdad44e4a5f069672f4a7b9f; no voters:
I20260809 23:14:07.655594 6249 raft_consensus.cc:2781] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:07.655665 6249 raft_consensus.cc:686] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 LEADER]: Becoming Leader. State: Replica: 4cb4fb59fdad44e4a5f069672f4a7b9f, State: Running, Role: LEADER
I20260809 23:14:07.655717 6249 consensus_queue.cc:237] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.655997 6252 tablet_replica.cc:439] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f: TxnStatusTablet state changed. Reason: New leader 4cb4fb59fdad44e4a5f069672f4a7b9f. Latest consensus state: current_term: 1 leader_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } }
I20260809 23:14:07.656086 6252 tablet_replica.cc:442] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:14:07.656276 5391 catalog_manager.cc:5495] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cb4fb59fdad44e4a5f069672f4a7b9f (127.2.76.193). New cstate: current_term: 1 leader_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } health_report { overall_health: UNKNOWN } } }
I20260809 23:14:07.656330 6275 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 23:14:07.656457 6275 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 23:14:07.661067 6266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.658694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:42953 (local address 127.0.0.1:60526)
0809 23:14:07.658755 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:07.658764 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:07.658804 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:07.658969 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:07.658969 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:07.658970 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:14:07.659008 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.659009 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.659736 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.659736 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.660255 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.660257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.660680 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.660680 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.660718 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.660730 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:14:07.660969 (+ 239us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:14:07.660974 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:14:07.660997 (+ 23us) client_negotiation.cc:240] Negotiation successful
0809 23:14:07.661025 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 23:14:07.661100 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.658755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60526 (local address 127.2.76.193:42953)
0809 23:14:07.658797 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.658799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.658807 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:07.658883 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.658884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.658898 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.658968 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:14:07.659042 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.659657 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.660327 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.660602 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.660666 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.660813 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:14:07.660907 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:14:07.660982 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.661003 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.661022 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:14:07.663045 6119 raft_consensus.cc:1261] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Refusing update from remote peer 4cb4fb59fdad44e4a5f069672f4a7b9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:07.663058 5880 raft_consensus.cc:1261] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Refusing update from remote peer 4cb4fb59fdad44e4a5f069672f4a7b9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:07.663322 6252 consensus_queue.cc:1035] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:07.663467 6249 consensus_queue.cc:1035] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:07.664454 6250 tablet_replica.cc:439] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0: TxnStatusTablet state changed. Reason: New leader 4cb4fb59fdad44e4a5f069672f4a7b9f. Latest consensus state: current_term: 1 leader_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } }
I20260809 23:14:07.664562 6250 tablet_replica.cc:442] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:14:07.664516 6248 tablet_replica.cc:439] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399: TxnStatusTablet state changed. Reason: New leader 4cb4fb59fdad44e4a5f069672f4a7b9f. Latest consensus state: current_term: 1 leader_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } }
I20260809 23:14:07.664600 6248 tablet_replica.cc:442] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:14:07.665577 6252 tablet_replica.cc:439] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } }
I20260809 23:14:07.665673 6252 tablet_replica.cc:442] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:14:07.665738 6281 mvcc.cc:205] Tried to move back new op lower bound from 7316755446425718784 to 7316755446398054400. Current Snapshot: MvccSnapshot[applied={T|T < 7316755446425718784}]
I20260809 23:14:07.666075 6252 tablet_replica.cc:439] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } }
I20260809 23:14:07.666150 6252 tablet_replica.cc:442] T e180bc8a520c49f6a0f48235f43778af P 4cb4fb59fdad44e4a5f069672f4a7b9f: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:14:07.666224 6248 tablet_replica.cc:439] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } }
I20260809 23:14:07.666296 6248 tablet_replica.cc:442] T e180bc8a520c49f6a0f48235f43778af P 0967cb4f28f24d14afa07e833eda7399: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:14:07.666872 6250 tablet_replica.cc:439] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } } }
I20260809 23:14:07.666947 6250 tablet_replica.cc:442] T e180bc8a520c49f6a0f48235f43778af P 1abdb315effc4e49a17e89efb29da6e0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:14:07.667037 6286 mvcc.cc:205] Tried to move back new op lower bound from 7316755446425718784 to 7316755446398054400. Current Snapshot: MvccSnapshot[applied={T|T < 7316755446425718784 or (T in {7316755446425718784})}]
I20260809 23:14:07.671532 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.669593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60538 (local address 127.2.76.193:42953)
0809 23:14:07.669629 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.669631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.669640 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:14:07.669706 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.669707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.669713 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.669775 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:14:07.669886 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.670275 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.670981 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.671159 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.671225 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.671356 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:14:07.671381 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:14:07.671418 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.671464 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.671478 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:14:07.675856 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.673969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:39419 (local address 127.2.76.193:42953)
0809 23:14:07.674020 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:07.674022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:07.674030 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:07.674072 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:07.674072 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:07.674079 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:07.674134 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:14:07.674250 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.674609 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.675245 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.675395 (+ 150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.675481 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.675650 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:14:07.675676 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:14:07.675735 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:07.675784 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 23:14:07.675797 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 23:14:07.675863 5739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:07.673883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:42953 (local address 127.2.76.193:39419)
0809 23:14:07.673977 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:07.673997 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:07.674033 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:07.674148 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:07.674149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:07.674150 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:14:07.674192 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.674193 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.674681 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.674682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.675170 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:07.675172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:07.675475 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:07.675479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:07.675539 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:07.675556 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:14:07.675735 (+ 179us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:14:07.675737 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:14:07.675777 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 23:14:07.675807 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 23:14:07.677716 6119 raft_consensus.cc:1261] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Refusing update from remote peer 4cb4fb59fdad44e4a5f069672f4a7b9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:07.677748 5880 raft_consensus.cc:1261] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Refusing update from remote peer 4cb4fb59fdad44e4a5f069672f4a7b9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:07.677906 6252 consensus_queue.cc:1035] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:07.678002 6249 consensus_queue.cc:1035] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:07.679482 6282 mvcc.cc:205] Tried to move back new op lower bound from 7316755446486720512 to 7316755446207959040. Current Snapshot: MvccSnapshot[applied={T|T < 7316755446486720512}]
I20260809 23:14:07.688791 5391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:07.688869 5391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:07.689450 5391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255: Sending DeleteTablet for 3 replicas of tablet fa771cb02c2149829ea8631878bac4fa
I20260809 23:14:07.689754 6080 tablet_service.cc:1514] Processing DeleteTablet for tablet fa771cb02c2149829ea8631878bac4fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:07 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:34992
I20260809 23:14:07.689951 5627 tablet_service.cc:1514] Processing DeleteTablet for tablet fa771cb02c2149829ea8631878bac4fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:07 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:60514
I20260809 23:14:07.689931 5391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:14:07.690069 5391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:14:07.690160 5850 tablet_service.cc:1514] Processing DeleteTablet for tablet fa771cb02c2149829ea8631878bac4fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:07 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:50588
I20260809 23:14:07.690351 6313 tablet_replica.cc:330] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399: stopping tablet replica
I20260809 23:14:07.690351 6321 tablet_replica.cc:330] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0: stopping tablet replica
I20260809 23:14:07.690440 6313 raft_consensus.cc:2229] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:07.690502 6313 raft_consensus.cc:2258] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:07.690594 6321 raft_consensus.cc:2229] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:07.690639 6315 tablet_replica.cc:330] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f: stopping tablet replica
I20260809 23:14:07.690733 6315 raft_consensus.cc:2229] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:14:07.690819 6315 raft_consensus.cc:2258] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:07.691115 6321 raft_consensus.cc:2258] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:07.691172 6315 ts_tablet_manager.cc:1905] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:07.691191 6080 tablet_service.cc:1467] Processing CreateTablet for tablet 1bcce9b3868640e28d1aa887b72cdec7 (DEFAULT_TABLE table=test-table [id=07483b390971431997777b14ec4e3ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:07.691237 5627 tablet_service.cc:1467] Processing CreateTablet for tablet 1bcce9b3868640e28d1aa887b72cdec7 (DEFAULT_TABLE table=test-table [id=07483b390971431997777b14ec4e3ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:07.691299 6080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bcce9b3868640e28d1aa887b72cdec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:07.691388 5627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bcce9b3868640e28d1aa887b72cdec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:07.692237 5850 tablet_service.cc:1467] Processing CreateTablet for tablet 1bcce9b3868640e28d1aa887b72cdec7 (DEFAULT_TABLE table=test-table [id=07483b390971431997777b14ec4e3ef2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:07.692338 5850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bcce9b3868640e28d1aa887b72cdec7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:07.692598 6237 tablet_bootstrap.cc:492] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399: Bootstrap starting.
I20260809 23:14:07.692768 6238 tablet_bootstrap.cc:492] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f: Bootstrap starting.
I20260809 23:14:07.693044 6237 tablet_bootstrap.cc:654] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:07.693217 6238 tablet_bootstrap.cc:654] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:07.693423 6239 tablet_bootstrap.cc:492] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0: Bootstrap starting.
I20260809 23:14:07.693610 6315 ts_tablet_manager.cc:1918] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:14:07.693691 6315 log.cc:1196] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/wal/wals/fa771cb02c2149829ea8631878bac4fa
I20260809 23:14:07.693708 6237 tablet_bootstrap.cc:492] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399: No bootstrap required, opened a new log
I20260809 23:14:07.693747 6237 ts_tablet_manager.cc:1397] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 23:14:07.693778 6239 tablet_bootstrap.cc:654] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:07.693878 6237 raft_consensus.cc:348] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.693938 6237 raft_consensus.cc:374] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:07.693938 6315 ts_tablet_manager.cc:1939] T fa771cb02c2149829ea8631878bac4fa P 4cb4fb59fdad44e4a5f069672f4a7b9f: Deleting consensus metadata
I20260809 23:14:07.693959 6237 raft_consensus.cc:729] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0967cb4f28f24d14afa07e833eda7399, State: Initialized, Role: FOLLOWER
I20260809 23:14:07.694020 6237 consensus_queue.cc:260] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.694128 6237 ts_tablet_manager.cc:1428] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 23:14:07.694371 5374 catalog_manager.cc:4841] TS 4cb4fb59fdad44e4a5f069672f4a7b9f (127.2.76.193:42953): tablet fa771cb02c2149829ea8631878bac4fa (table test-table [id=f901749fb4dd495ab234fe49e17880f9]) successfully deleted
I20260809 23:14:07.694538 6239 tablet_bootstrap.cc:492] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0: No bootstrap required, opened a new log
I20260809 23:14:07.694577 6239 ts_tablet_manager.cc:1397] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 23:14:07.694696 6239 raft_consensus.cc:348] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.694753 6239 raft_consensus.cc:374] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:07.694772 6239 raft_consensus.cc:729] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1abdb315effc4e49a17e89efb29da6e0, State: Initialized, Role: FOLLOWER
I20260809 23:14:07.694813 6239 consensus_queue.cc:260] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.694900 6239 ts_tablet_manager.cc:1428] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:07.696487 6313 ts_tablet_manager.cc:1905] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:07.696525 6321 ts_tablet_manager.cc:1905] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:07.697672 6313 ts_tablet_manager.cc:1918] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:14:07.697741 6313 log.cc:1196] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/fa771cb02c2149829ea8631878bac4fa
I20260809 23:14:07.697803 6238 tablet_bootstrap.cc:492] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f: No bootstrap required, opened a new log
I20260809 23:14:07.697842 6238 ts_tablet_manager.cc:1397] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260809 23:14:07.697856 6321 ts_tablet_manager.cc:1918] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:14:07.697916 6321 log.cc:1196] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/fa771cb02c2149829ea8631878bac4fa
I20260809 23:14:07.698004 6313 ts_tablet_manager.cc:1939] T fa771cb02c2149829ea8631878bac4fa P 0967cb4f28f24d14afa07e833eda7399: Deleting consensus metadata
I20260809 23:14:07.697978 6238 raft_consensus.cc:348] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.698060 6238 raft_consensus.cc:374] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:07.698084 6238 raft_consensus.cc:729] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb4fb59fdad44e4a5f069672f4a7b9f, State: Initialized, Role: FOLLOWER
I20260809 23:14:07.698127 6238 consensus_queue.cc:260] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.698149 6321 ts_tablet_manager.cc:1939] T fa771cb02c2149829ea8631878bac4fa P 1abdb315effc4e49a17e89efb29da6e0: Deleting consensus metadata
I20260809 23:14:07.698197 6238 ts_tablet_manager.cc:1428] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:07.698482 5370 catalog_manager.cc:4841] TS 0967cb4f28f24d14afa07e833eda7399 (127.2.76.195:37023): tablet fa771cb02c2149829ea8631878bac4fa (table test-table [id=f901749fb4dd495ab234fe49e17880f9]) successfully deleted
I20260809 23:14:07.698563 5374 catalog_manager.cc:4841] TS 1abdb315effc4e49a17e89efb29da6e0 (127.2.76.194:33169): tablet fa771cb02c2149829ea8631878bac4fa (table test-table [id=f901749fb4dd495ab234fe49e17880f9]) successfully deleted
I20260809 23:14:07.712001 6249 raft_consensus.cc:482] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:14:07.712064 6249 raft_consensus.cc:504] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.712195 6249 leader_election.cc:288] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1abdb315effc4e49a17e89efb29da6e0 (127.2.76.194:33169), 0967cb4f28f24d14afa07e833eda7399 (127.2.76.195:37023)
I20260809 23:14:07.712317 6119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bcce9b3868640e28d1aa887b72cdec7" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0967cb4f28f24d14afa07e833eda7399" is_pre_election: true
I20260809 23:14:07.712329 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bcce9b3868640e28d1aa887b72cdec7" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1abdb315effc4e49a17e89efb29da6e0" is_pre_election: true
I20260809 23:14:07.712404 5880 raft_consensus.cc:2454] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 0.
I20260809 23:14:07.712405 6119 raft_consensus.cc:2454] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 0.
I20260809 23:14:07.712543 5545 leader_election.cc:302] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0967cb4f28f24d14afa07e833eda7399, 4cb4fb59fdad44e4a5f069672f4a7b9f; no voters:
I20260809 23:14:07.712646 6249 raft_consensus.cc:2781] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:14:07.712690 6249 raft_consensus.cc:482] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:14:07.712718 6249 raft_consensus.cc:3037] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:07.713150 6249 raft_consensus.cc:504] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.713282 6249 leader_election.cc:288] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 election: Requested vote from peers 1abdb315effc4e49a17e89efb29da6e0 (127.2.76.194:33169), 0967cb4f28f24d14afa07e833eda7399 (127.2.76.195:37023)
I20260809 23:14:07.713379 5880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bcce9b3868640e28d1aa887b72cdec7" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1abdb315effc4e49a17e89efb29da6e0"
I20260809 23:14:07.713454 5880 raft_consensus.cc:3037] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:07.713430 6119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bcce9b3868640e28d1aa887b72cdec7" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0967cb4f28f24d14afa07e833eda7399"
I20260809 23:14:07.713486 6119 raft_consensus.cc:3037] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:07.713967 5880 raft_consensus.cc:2454] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 1.
I20260809 23:14:07.714042 6119 raft_consensus.cc:2454] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 1.
I20260809 23:14:07.714133 5548 leader_election.cc:302] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1abdb315effc4e49a17e89efb29da6e0, 4cb4fb59fdad44e4a5f069672f4a7b9f; no voters:
I20260809 23:14:07.714221 6249 raft_consensus.cc:2781] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:07.714267 6249 raft_consensus.cc:686] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 LEADER]: Becoming Leader. State: Replica: 4cb4fb59fdad44e4a5f069672f4a7b9f, State: Running, Role: LEADER
I20260809 23:14:07.714316 6249 consensus_queue.cc:237] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:07.714753 5391 catalog_manager.cc:5495] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cb4fb59fdad44e4a5f069672f4a7b9f (127.2.76.193). New cstate: current_term: 1 leader_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } health_report { overall_health: UNKNOWN } } }
W20260809 23:14:07.722205 5730 tablet.cc:2369] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:14:07.725339 6119 raft_consensus.cc:1261] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Refusing update from remote peer 4cb4fb59fdad44e4a5f069672f4a7b9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:07.725412 5880 raft_consensus.cc:1261] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Refusing update from remote peer 4cb4fb59fdad44e4a5f069672f4a7b9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:07.725510 6252 consensus_queue.cc:1035] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:07.725591 6249 consensus_queue.cc:1035] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 23:14:07.729871 6281 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 23:14:09.246565 5390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:09.246657 5390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:09.247524 5390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255: Sending DeleteTablet for 3 replicas of tablet 1bcce9b3868640e28d1aa887b72cdec7
I20260809 23:14:09.247828 5850 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bcce9b3868640e28d1aa887b72cdec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:09 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:50588
I20260809 23:14:09.247848 5627 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bcce9b3868640e28d1aa887b72cdec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:09 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:60514
I20260809 23:14:09.247871 6080 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bcce9b3868640e28d1aa887b72cdec7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:09 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:34992
I20260809 23:14:09.248085 6369 tablet_replica.cc:330] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f: stopping tablet replica
I20260809 23:14:09.248172 6369 raft_consensus.cc:2229] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:14:09.248231 6369 raft_consensus.cc:2258] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:09.248468 6369 ts_tablet_manager.cc:1905] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:09.250417 6369 ts_tablet_manager.cc:1918] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 23:14:09.250483 6369 log.cc:1196] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/wal/wals/1bcce9b3868640e28d1aa887b72cdec7
I20260809 23:14:09.250723 6369 ts_tablet_manager.cc:1939] T 1bcce9b3868640e28d1aa887b72cdec7 P 4cb4fb59fdad44e4a5f069672f4a7b9f: Deleting consensus metadata
I20260809 23:14:09.251027 5374 catalog_manager.cc:4841] TS 4cb4fb59fdad44e4a5f069672f4a7b9f (127.2.76.193:42953): tablet 1bcce9b3868640e28d1aa887b72cdec7 (table test-table [id=07483b390971431997777b14ec4e3ef2]) successfully deleted
I20260809 23:14:09.251240 6368 tablet_replica.cc:330] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399: stopping tablet replica
I20260809 23:14:09.251322 6368 raft_consensus.cc:2229] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:09.251362 6368 raft_consensus.cc:2258] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:09.251444 6365 tablet_replica.cc:330] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0: stopping tablet replica
I20260809 23:14:09.251499 6365 raft_consensus.cc:2229] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:09.251540 6365 raft_consensus.cc:2258] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:09.251590 6368 ts_tablet_manager.cc:1905] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:09.251777 6365 ts_tablet_manager.cc:1905] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:09.253602 6365 ts_tablet_manager.cc:1918] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 23:14:09.253669 6365 log.cc:1196] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/1bcce9b3868640e28d1aa887b72cdec7
I20260809 23:14:09.253674 6368 ts_tablet_manager.cc:1918] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 23:14:09.253731 6368 log.cc:1196] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/1bcce9b3868640e28d1aa887b72cdec7
I20260809 23:14:09.253933 6365 ts_tablet_manager.cc:1939] T 1bcce9b3868640e28d1aa887b72cdec7 P 1abdb315effc4e49a17e89efb29da6e0: Deleting consensus metadata
I20260809 23:14:09.253943 6368 ts_tablet_manager.cc:1939] T 1bcce9b3868640e28d1aa887b72cdec7 P 0967cb4f28f24d14afa07e833eda7399: Deleting consensus metadata
I20260809 23:14:09.254251 5370 catalog_manager.cc:4841] TS 0967cb4f28f24d14afa07e833eda7399 (127.2.76.195:37023): tablet 1bcce9b3868640e28d1aa887b72cdec7 (table test-table [id=07483b390971431997777b14ec4e3ef2]) successfully deleted
I20260809 23:14:09.254315 5374 catalog_manager.cc:4841] TS 1abdb315effc4e49a17e89efb29da6e0 (127.2.76.194:33169): tablet 1bcce9b3868640e28d1aa887b72cdec7 (table test-table [id=07483b390971431997777b14ec4e3ef2]) successfully deleted
Aug 09 23:14:09 dist-test-slave-7vjv krb5kdc[5320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317249, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 23:14:09 dist-test-slave-7vjv krb5kdc[5320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317249, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:09.271682 6383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:09.265122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58572 (local address 127.2.76.254:34155)
0809 23:14:09.265261 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:09.265263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:09.265272 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:14:09.265294 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:09.265294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:09.265305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:09.265392 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:09.265915 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:09.266389 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:09.267113 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:09.267338 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:09.267407 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:09.269481 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:09.269482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:09.269482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:09.269488 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:09.270736 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:09.271089 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:09.271090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:09.271090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:09.271147 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:09.271315 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:09.271315 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:09.271315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:09.271411 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:09.271485 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:09.271559 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 23:14:09.271624 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":59,"threads_started":1}
I20260809 23:14:09.273017 5390 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:14:09.273166 5390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:14:09.274397 5627 tablet_service.cc:1467] Processing CreateTablet for tablet afaed89243b14db18f9d0c3b2bf952b4 (DEFAULT_TABLE table=test-table [id=e2209c4008544ec7ab9bb47bff4a7766]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:09.274401 5850 tablet_service.cc:1467] Processing CreateTablet for tablet afaed89243b14db18f9d0c3b2bf952b4 (DEFAULT_TABLE table=test-table [id=e2209c4008544ec7ab9bb47bff4a7766]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:09.274401 6080 tablet_service.cc:1467] Processing CreateTablet for tablet afaed89243b14db18f9d0c3b2bf952b4 (DEFAULT_TABLE table=test-table [id=e2209c4008544ec7ab9bb47bff4a7766]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:09.274544 6080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afaed89243b14db18f9d0c3b2bf952b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:09.274542 5850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afaed89243b14db18f9d0c3b2bf952b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:09.274549 5627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afaed89243b14db18f9d0c3b2bf952b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:09.275568 6384 tablet_bootstrap.cc:492] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f: Bootstrap starting.
I20260809 23:14:09.275874 6387 tablet_bootstrap.cc:492] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0: Bootstrap starting.
I20260809 23:14:09.275923 6386 tablet_bootstrap.cc:492] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399: Bootstrap starting.
I20260809 23:14:09.276057 6384 tablet_bootstrap.cc:654] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:09.276311 6386 tablet_bootstrap.cc:654] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:09.276521 6387 tablet_bootstrap.cc:654] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:09.276749 6384 tablet_bootstrap.cc:492] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f: No bootstrap required, opened a new log
I20260809 23:14:09.276806 6384 ts_tablet_manager.cc:1397] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:09.276917 6386 tablet_bootstrap.cc:492] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399: No bootstrap required, opened a new log
I20260809 23:14:09.276928 6384 raft_consensus.cc:348] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.276984 6386 ts_tablet_manager.cc:1397] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:09.277009 6384 raft_consensus.cc:374] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:09.277038 6384 raft_consensus.cc:729] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb4fb59fdad44e4a5f069672f4a7b9f, State: Initialized, Role: FOLLOWER
I20260809 23:14:09.277094 6384 consensus_queue.cc:260] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.277158 6386 raft_consensus.cc:348] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.277231 6386 raft_consensus.cc:374] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:09.277221 6384 ts_tablet_manager.cc:1428] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:09.277254 6386 raft_consensus.cc:729] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0967cb4f28f24d14afa07e833eda7399, State: Initialized, Role: FOLLOWER
I20260809 23:14:09.277310 6386 consensus_queue.cc:260] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.277412 6387 tablet_bootstrap.cc:492] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0: No bootstrap required, opened a new log
I20260809 23:14:09.277458 6387 ts_tablet_manager.cc:1397] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:14:09.277482 6386 ts_tablet_manager.cc:1428] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:09.277560 6387 raft_consensus.cc:348] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.277627 6387 raft_consensus.cc:374] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:09.277647 6387 raft_consensus.cc:729] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1abdb315effc4e49a17e89efb29da6e0, State: Initialized, Role: FOLLOWER
I20260809 23:14:09.277702 6387 consensus_queue.cc:260] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.277866 6387 ts_tablet_manager.cc:1428] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 23:14:09.310501 6249 raft_consensus.cc:482] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:14:09.310561 6249 raft_consensus.cc:504] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.310672 6249 leader_election.cc:288] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1abdb315effc4e49a17e89efb29da6e0 (127.2.76.194:33169), 0967cb4f28f24d14afa07e833eda7399 (127.2.76.195:37023)
I20260809 23:14:09.310849 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afaed89243b14db18f9d0c3b2bf952b4" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1abdb315effc4e49a17e89efb29da6e0" is_pre_election: true
I20260809 23:14:09.310881 6119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afaed89243b14db18f9d0c3b2bf952b4" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0967cb4f28f24d14afa07e833eda7399" is_pre_election: true
I20260809 23:14:09.310938 5879 raft_consensus.cc:2454] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 0.
I20260809 23:14:09.310941 6119 raft_consensus.cc:2454] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 0.
I20260809 23:14:09.311122 5545 leader_election.cc:302] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0967cb4f28f24d14afa07e833eda7399, 4cb4fb59fdad44e4a5f069672f4a7b9f; no voters:
I20260809 23:14:09.311256 6249 raft_consensus.cc:2781] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:14:09.311313 6249 raft_consensus.cc:482] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:14:09.311327 6249 raft_consensus.cc:3037] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:09.311816 6249 raft_consensus.cc:504] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.311944 6249 leader_election.cc:288] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 election: Requested vote from peers 1abdb315effc4e49a17e89efb29da6e0 (127.2.76.194:33169), 0967cb4f28f24d14afa07e833eda7399 (127.2.76.195:37023)
I20260809 23:14:09.312114 6119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afaed89243b14db18f9d0c3b2bf952b4" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0967cb4f28f24d14afa07e833eda7399"
I20260809 23:14:09.312134 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afaed89243b14db18f9d0c3b2bf952b4" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1abdb315effc4e49a17e89efb29da6e0"
I20260809 23:14:09.312183 5879 raft_consensus.cc:3037] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:09.312184 6119 raft_consensus.cc:3037] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:09.312767 5879 raft_consensus.cc:2454] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 1.
I20260809 23:14:09.312767 6119 raft_consensus.cc:2454] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 1.
I20260809 23:14:09.312927 5548 leader_election.cc:302] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1abdb315effc4e49a17e89efb29da6e0, 4cb4fb59fdad44e4a5f069672f4a7b9f; no voters:
I20260809 23:14:09.313051 6249 raft_consensus.cc:2781] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:09.313109 6249 raft_consensus.cc:686] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 LEADER]: Becoming Leader. State: Replica: 4cb4fb59fdad44e4a5f069672f4a7b9f, State: Running, Role: LEADER
I20260809 23:14:09.313145 6249 consensus_queue.cc:237] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.313668 5391 catalog_manager.cc:5495] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cb4fb59fdad44e4a5f069672f4a7b9f (127.2.76.193). New cstate: current_term: 1 leader_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } health_report { overall_health: UNKNOWN } } }
I20260809 23:14:09.325052 6406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:09.322666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60548 (local address 127.2.76.193:42953)
0809 23:14:09.322791 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:09.322793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:09.322799 (+ 6us) server_negotiation.cc:407] Connection header received
0809 23:14:09.322818 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:09.322819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:09.322828 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:09.322905 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:14:09.322998 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:09.323501 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:09.324360 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:09.324579 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:09.324650 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:09.324783 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:14:09.324815 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:14:09.324911 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:09.324960 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 23:14:09.324981 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260809 23:14:09.326817 5879 raft_consensus.cc:1261] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Refusing update from remote peer 4cb4fb59fdad44e4a5f069672f4a7b9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:09.326846 6119 raft_consensus.cc:1261] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Refusing update from remote peer 4cb4fb59fdad44e4a5f069672f4a7b9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:09.327062 6409 consensus_queue.cc:1035] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:09.327148 6343 consensus_queue.cc:1035] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:09.336683 5391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:09.336753 5391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:09.337193 5391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255: Sending DeleteTablet for 3 replicas of tablet afaed89243b14db18f9d0c3b2bf952b4
I20260809 23:14:09.337396 5627 tablet_service.cc:1514] Processing DeleteTablet for tablet afaed89243b14db18f9d0c3b2bf952b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:09 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:60514
I20260809 23:14:09.337396 6080 tablet_service.cc:1514] Processing DeleteTablet for tablet afaed89243b14db18f9d0c3b2bf952b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:09 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:34992
I20260809 23:14:09.337486 5850 tablet_service.cc:1514] Processing DeleteTablet for tablet afaed89243b14db18f9d0c3b2bf952b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:09 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:50588
I20260809 23:14:09.337565 6368 tablet_replica.cc:330] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399: stopping tablet replica
I20260809 23:14:09.337613 6368 raft_consensus.cc:2229] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:09.337661 6368 raft_consensus.cc:2258] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:09.337616 5391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 23:14:09.337725 6369 tablet_replica.cc:330] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f: stopping tablet replica
W20260809 23:14:09.337738 5391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:14:09.337769 6369 raft_consensus.cc:2229] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:14:09.337808 6369 raft_consensus.cc:2258] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:09.337867 6365 tablet_replica.cc:330] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0: stopping tablet replica
I20260809 23:14:09.337918 6365 raft_consensus.cc:2229] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:09.337958 6365 raft_consensus.cc:2258] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:09.338073 6369 ts_tablet_manager.cc:1905] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:09.338203 6365 ts_tablet_manager.cc:1905] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:09.338238 6368 ts_tablet_manager.cc:1905] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:09.339422 5627 tablet_service.cc:1467] Processing CreateTablet for tablet f5d9b9530e7b4297b4fd0742e9d979fb (DEFAULT_TABLE table=test-table [id=b44104a27a8e4fbba7d3364e7f2321c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:09.339473 5850 tablet_service.cc:1467] Processing CreateTablet for tablet f5d9b9530e7b4297b4fd0742e9d979fb (DEFAULT_TABLE table=test-table [id=b44104a27a8e4fbba7d3364e7f2321c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:09.339516 6080 tablet_service.cc:1467] Processing CreateTablet for tablet f5d9b9530e7b4297b4fd0742e9d979fb (DEFAULT_TABLE table=test-table [id=b44104a27a8e4fbba7d3364e7f2321c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:09.339570 6368 ts_tablet_manager.cc:1918] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:14:09.339610 6080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5d9b9530e7b4297b4fd0742e9d979fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:09.339617 6368 log.cc:1196] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/afaed89243b14db18f9d0c3b2bf952b4
I20260809 23:14:09.339625 6369 ts_tablet_manager.cc:1918] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:14:09.339679 6369 log.cc:1196] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-0/wal/wals/afaed89243b14db18f9d0c3b2bf952b4
I20260809 23:14:09.339696 6365 ts_tablet_manager.cc:1918] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:14:09.339737 6365 log.cc:1196] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/afaed89243b14db18f9d0c3b2bf952b4
I20260809 23:14:09.339855 6368 ts_tablet_manager.cc:1939] T afaed89243b14db18f9d0c3b2bf952b4 P 0967cb4f28f24d14afa07e833eda7399: Deleting consensus metadata
I20260809 23:14:09.339905 6369 ts_tablet_manager.cc:1939] T afaed89243b14db18f9d0c3b2bf952b4 P 4cb4fb59fdad44e4a5f069672f4a7b9f: Deleting consensus metadata
I20260809 23:14:09.339946 6365 ts_tablet_manager.cc:1939] T afaed89243b14db18f9d0c3b2bf952b4 P 1abdb315effc4e49a17e89efb29da6e0: Deleting consensus metadata
I20260809 23:14:09.340034 5627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5d9b9530e7b4297b4fd0742e9d979fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:09.340072 5850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5d9b9530e7b4297b4fd0742e9d979fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:09.340216 5370 catalog_manager.cc:4841] TS 0967cb4f28f24d14afa07e833eda7399 (127.2.76.195:37023): tablet afaed89243b14db18f9d0c3b2bf952b4 (table test-table [id=e2209c4008544ec7ab9bb47bff4a7766]) successfully deleted
I20260809 23:14:09.340726 6386 tablet_bootstrap.cc:492] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399: Bootstrap starting.
I20260809 23:14:09.341074 6386 tablet_bootstrap.cc:654] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:09.341223 6387 tablet_bootstrap.cc:492] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0: Bootstrap starting.
I20260809 23:14:09.341229 6384 tablet_bootstrap.cc:492] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f: Bootstrap starting.
I20260809 23:14:09.341535 5374 catalog_manager.cc:4841] TS 4cb4fb59fdad44e4a5f069672f4a7b9f (127.2.76.193:42953): tablet afaed89243b14db18f9d0c3b2bf952b4 (table test-table [id=e2209c4008544ec7ab9bb47bff4a7766]) successfully deleted
I20260809 23:14:09.341564 6384 tablet_bootstrap.cc:654] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:09.341647 6387 tablet_bootstrap.cc:654] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:09.342082 6384 tablet_bootstrap.cc:492] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f: No bootstrap required, opened a new log
I20260809 23:14:09.342082 6386 tablet_bootstrap.cc:492] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399: No bootstrap required, opened a new log
I20260809 23:14:09.342131 6384 ts_tablet_manager.cc:1397] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:09.342131 6386 ts_tablet_manager.cc:1397] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:09.342247 6384 raft_consensus.cc:348] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.342315 6384 raft_consensus.cc:374] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:09.342286 6386 raft_consensus.cc:348] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.342336 6384 raft_consensus.cc:729] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb4fb59fdad44e4a5f069672f4a7b9f, State: Initialized, Role: FOLLOWER
I20260809 23:14:09.342345 6386 raft_consensus.cc:374] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:09.342366 6386 raft_consensus.cc:729] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0967cb4f28f24d14afa07e833eda7399, State: Initialized, Role: FOLLOWER
I20260809 23:14:09.342402 6387 tablet_bootstrap.cc:492] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0: No bootstrap required, opened a new log
I20260809 23:14:09.342386 6384 consensus_queue.cc:260] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.342411 6386 consensus_queue.cc:260] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.342453 6387 ts_tablet_manager.cc:1397] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 23:14:09.342479 6384 ts_tablet_manager.cc:1428] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:09.342496 6386 ts_tablet_manager.cc:1428] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:09.342657 6387 raft_consensus.cc:348] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.342734 6387 raft_consensus.cc:374] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:09.342759 6387 raft_consensus.cc:729] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1abdb315effc4e49a17e89efb29da6e0, State: Initialized, Role: FOLLOWER
I20260809 23:14:09.342824 6387 consensus_queue.cc:260] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.342907 6387 ts_tablet_manager.cc:1428] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 23:14:09.343705 5374 catalog_manager.cc:4841] TS 1abdb315effc4e49a17e89efb29da6e0 (127.2.76.194:33169): tablet afaed89243b14db18f9d0c3b2bf952b4 (table test-table [id=e2209c4008544ec7ab9bb47bff4a7766]) successfully deleted
I20260809 23:14:09.457923 6249 raft_consensus.cc:482] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:14:09.458009 6249 raft_consensus.cc:504] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.458153 6249 leader_election.cc:288] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1abdb315effc4e49a17e89efb29da6e0 (127.2.76.194:33169), 0967cb4f28f24d14afa07e833eda7399 (127.2.76.195:37023)
I20260809 23:14:09.458350 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5d9b9530e7b4297b4fd0742e9d979fb" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1abdb315effc4e49a17e89efb29da6e0" is_pre_election: true
I20260809 23:14:09.458371 6119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5d9b9530e7b4297b4fd0742e9d979fb" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0967cb4f28f24d14afa07e833eda7399" is_pre_election: true
I20260809 23:14:09.458460 5879 raft_consensus.cc:2454] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 0.
I20260809 23:14:09.458460 6119 raft_consensus.cc:2454] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 0.
I20260809 23:14:09.458623 5545 leader_election.cc:302] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0967cb4f28f24d14afa07e833eda7399, 4cb4fb59fdad44e4a5f069672f4a7b9f; no voters:
I20260809 23:14:09.458720 6249 raft_consensus.cc:2781] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:14:09.458760 6249 raft_consensus.cc:482] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:14:09.458787 6249 raft_consensus.cc:3037] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:09.459367 6249 raft_consensus.cc:504] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.459492 6249 leader_election.cc:288] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 election: Requested vote from peers 1abdb315effc4e49a17e89efb29da6e0 (127.2.76.194:33169), 0967cb4f28f24d14afa07e833eda7399 (127.2.76.195:37023)
I20260809 23:14:09.459645 5879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5d9b9530e7b4297b4fd0742e9d979fb" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1abdb315effc4e49a17e89efb29da6e0"
I20260809 23:14:09.459730 5879 raft_consensus.cc:3037] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:09.459686 6119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5d9b9530e7b4297b4fd0742e9d979fb" candidate_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0967cb4f28f24d14afa07e833eda7399"
I20260809 23:14:09.459787 6119 raft_consensus.cc:3037] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:09.460391 5879 raft_consensus.cc:2454] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 1.
I20260809 23:14:09.460464 6119 raft_consensus.cc:2454] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cb4fb59fdad44e4a5f069672f4a7b9f in term 1.
I20260809 23:14:09.460570 5548 leader_election.cc:302] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1abdb315effc4e49a17e89efb29da6e0, 4cb4fb59fdad44e4a5f069672f4a7b9f; no voters:
I20260809 23:14:09.460701 6249 raft_consensus.cc:2781] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:09.460772 6249 raft_consensus.cc:686] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [term 1 LEADER]: Becoming Leader. State: Replica: 4cb4fb59fdad44e4a5f069672f4a7b9f, State: Running, Role: LEADER
I20260809 23:14:09.460834 6249 consensus_queue.cc:237] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } }
I20260809 23:14:09.461367 5389 catalog_manager.cc:5495] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cb4fb59fdad44e4a5f069672f4a7b9f (127.2.76.193). New cstate: current_term: 1 leader_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb4fb59fdad44e4a5f069672f4a7b9f" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 42953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 } health_report { overall_health: UNKNOWN } } }
I20260809 23:14:09.490070 6119 raft_consensus.cc:1261] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Refusing update from remote peer 4cb4fb59fdad44e4a5f069672f4a7b9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:09.490160 5879 raft_consensus.cc:1261] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Refusing update from remote peer 4cb4fb59fdad44e4a5f069672f4a7b9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:09.490267 6336 consensus_queue.cc:1035] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0967cb4f28f24d14afa07e833eda7399" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 37023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:09.490526 6336 consensus_queue.cc:1035] T f5d9b9530e7b4297b4fd0742e9d979fb P 4cb4fb59fdad44e4a5f069672f4a7b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1abdb315effc4e49a17e89efb29da6e0" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 23:14:09.494314 6332 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 23:14:11.011090 5389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:11.011199 5389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:11.012012 5389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3581bfe8026747c6885a2abb39e76255: Sending DeleteTablet for 3 replicas of tablet f5d9b9530e7b4297b4fd0742e9d979fb
I20260809 23:14:11.012239 5628 tablet_service.cc:1514] Processing DeleteTablet for tablet f5d9b9530e7b4297b4fd0742e9d979fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:11 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:60514
I20260809 23:14:11.012252 5850 tablet_service.cc:1514] Processing DeleteTablet for tablet f5d9b9530e7b4297b4fd0742e9d979fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:11 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:50588
I20260809 23:14:11.012328 6080 tablet_service.cc:1514] Processing DeleteTablet for tablet f5d9b9530e7b4297b4fd0742e9d979fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:11 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:34992
I20260809 23:14:11.012439 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 5520
I20260809 23:14:11.012529 6465 tablet_replica.cc:330] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399: stopping tablet replica
I20260809 23:14:11.012583 6465 raft_consensus.cc:2229] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:11.012614 6465 raft_consensus.cc:2258] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:11.017331 6465 ts_tablet_manager.cc:1905] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:11.018210 6463 tablet_replica.cc:330] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0: stopping tablet replica
I20260809 23:14:11.018276 6463 raft_consensus.cc:2229] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:11.018318 6463 raft_consensus.cc:2258] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:11.019217 6463 ts_tablet_manager.cc:1905] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:11.020609 6465 ts_tablet_manager.cc:1918] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 23:14:11.020658 6465 log.cc:1196] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/f5d9b9530e7b4297b4fd0742e9d979fb
I20260809 23:14:11.020838 6465 ts_tablet_manager.cc:1939] T f5d9b9530e7b4297b4fd0742e9d979fb P 0967cb4f28f24d14afa07e833eda7399: Deleting consensus metadata
I20260809 23:14:11.021225 5370 catalog_manager.cc:4841] TS 0967cb4f28f24d14afa07e833eda7399 (127.2.76.195:37023): tablet f5d9b9530e7b4297b4fd0742e9d979fb (table test-table [id=b44104a27a8e4fbba7d3364e7f2321c9]) successfully deleted
I20260809 23:14:11.021404 6463 ts_tablet_manager.cc:1918] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 23:14:11.021483 6463 log.cc:1196] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/f5d9b9530e7b4297b4fd0742e9d979fb
I20260809 23:14:11.021816 6463 ts_tablet_manager.cc:1939] T f5d9b9530e7b4297b4fd0742e9d979fb P 1abdb315effc4e49a17e89efb29da6e0: Deleting consensus metadata
I20260809 23:14:11.022179 5374 catalog_manager.cc:4841] TS 1abdb315effc4e49a17e89efb29da6e0 (127.2.76.194:33169): tablet f5d9b9530e7b4297b4fd0742e9d979fb (table test-table [id=b44104a27a8e4fbba7d3364e7f2321c9]) successfully deleted
W20260809 23:14:11.028431 5374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:42953 (error 108)
I20260809 23:14:11.028729 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 5742
I20260809 23:14:11.030205 6470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:11.028787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:42953 (local address 127.0.0.1:55274)
0809 23:14:11.030117 (+ 1330us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:11.030164 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:42953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1307,"thread_start_us":52,"threads_started":1}
W20260809 23:14:11.030283 5374 catalog_manager.cc:4584] TS 4cb4fb59fdad44e4a5f069672f4a7b9f (127.2.76.193:42953): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f5d9b9530e7b4297b4fd0742e9d979fb: Network error: Client connection negotiation failed: client connection to 127.2.76.193:42953: connect: Connection refused (error 111)
I20260809 23:14:11.036475 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 5975
I20260809 23:14:11.051587 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 5338
2026-08-09T23:14:11Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5400 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:14:11 dist-test-slave-7vjv krb5kdc[6473](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:14:11 dist-test-slave-7vjv krb5kdc[6473](info): set up 2 sockets
Aug 09 23:14:11 dist-test-slave-7vjv krb5kdc[6473](info): commencing operation
krb5kdc: starting...
W20260809 23:14:12.216405 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:14:12 dist-test-slave-7vjv krb5kdc[6473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:14:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:14:12.324139 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:39003
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:32823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:39003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:12.403399 6491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:12.403594 6491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:12.403630 6491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:12.404695 6491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:12.404752 6491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:12.404773 6491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:12.404800 6491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:12.404821 6491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:14:12.406165 6491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:32823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:39003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:39003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:12.406417 6491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:12.406633 6491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:12.410981 6505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:12.411031 6501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:12.411006 6500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:12.411121 6491 server_base.cc:1034] running on GCE node
I20260809 23:14:12.411396 6491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:12.411648 6491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:12.412815 6491 hybrid_clock.cc:648] HybridClock initialized: now 1786317252412812 us; error 35 us; skew 500 ppm
Aug 09 23:14:12 dist-test-slave-7vjv krb5kdc[6473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317252, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:12.420377 6491 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:14:12.420780 6491 webserver.cc:460] Webserver started at http://127.2.76.254:36183/ using document root <none> and password file <none>
I20260809 23:14:12.420994 6491 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:12.421043 6491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:12.421182 6491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:12.422128 6491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "2b87a1d50bb54d21b0e5eee62e9b6f9f"
format_stamp: "Formatted at 2026-08-09 23:14:12 on dist-test-slave-7vjv"
server_key: "dafe58ffb042c6898ccd950ca118b095"
server_key_iv: "297310024809c8a7df15b18077f4f544"
server_key_version: "encryptionkey@0"
I20260809 23:14:12.422490 6491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "2b87a1d50bb54d21b0e5eee62e9b6f9f"
format_stamp: "Formatted at 2026-08-09 23:14:12 on dist-test-slave-7vjv"
server_key: "dafe58ffb042c6898ccd950ca118b095"
server_key_iv: "297310024809c8a7df15b18077f4f544"
server_key_version: "encryptionkey@0"
I20260809 23:14:12.423717 6491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:14:12.424420 6515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:12.424575 6491 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:12.424655 6491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "2b87a1d50bb54d21b0e5eee62e9b6f9f"
format_stamp: "Formatted at 2026-08-09 23:14:12 on dist-test-slave-7vjv"
server_key: "dafe58ffb042c6898ccd950ca118b095"
server_key_iv: "297310024809c8a7df15b18077f4f544"
server_key_version: "encryptionkey@0"
I20260809 23:14:12.424702 6491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:12.437006 6491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:12.438462 6491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:12.438561 6491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:12.441946 6491 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:39003
I20260809 23:14:12.442011 6602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:39003 every 8 connection(s)
I20260809 23:14:12.442271 6491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:12.443039 6603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:12.445001 6603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f: Bootstrap starting.
I20260809 23:14:12.445601 6603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:12.445899 6603 log.cc:824] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:12.446565 6603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f: No bootstrap required, opened a new log
I20260809 23:14:12.448145 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 6491
I20260809 23:14:12.448212 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:12.448241 6603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b87a1d50bb54d21b0e5eee62e9b6f9f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39003 } }
I20260809 23:14:12.448377 2355 external_mini_cluster.cc:1428] Setting key f0d472d59a68eca3a6e7bf268b329abf
I20260809 23:14:12.448390 6603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:12.448433 6603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b87a1d50bb54d21b0e5eee62e9b6f9f, State: Initialized, Role: FOLLOWER
I20260809 23:14:12.448570 6603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b87a1d50bb54d21b0e5eee62e9b6f9f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39003 } }
I20260809 23:14:12.448688 6603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:12.448725 6603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:12.448828 6603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:12.449383 6603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b87a1d50bb54d21b0e5eee62e9b6f9f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39003 } }
I20260809 23:14:12.449496 6603 leader_election.cc:302] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b87a1d50bb54d21b0e5eee62e9b6f9f; no voters:
I20260809 23:14:12.449695 6603 leader_election.cc:288] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:12.449750 6612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:12.449890 6612 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [term 1 LEADER]: Becoming Leader. State: Replica: 2b87a1d50bb54d21b0e5eee62e9b6f9f, State: Running, Role: LEADER
I20260809 23:14:12.449946 6603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:12.450012 6612 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b87a1d50bb54d21b0e5eee62e9b6f9f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39003 } }
I20260809 23:14:12.450392 6612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b87a1d50bb54d21b0e5eee62e9b6f9f. Latest consensus state: current_term: 1 leader_uuid: "2b87a1d50bb54d21b0e5eee62e9b6f9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b87a1d50bb54d21b0e5eee62e9b6f9f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39003 } } }
I20260809 23:14:12.450626 6612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:12.450562 6613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b87a1d50bb54d21b0e5eee62e9b6f9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b87a1d50bb54d21b0e5eee62e9b6f9f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39003 } } }
I20260809 23:14:12.451040 6613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:12.451174 6623 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:12.451566 6623 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:12.453681 6623 catalog_manager.cc:1347] Generated new cluster ID: 508320269f7a4cb59100c8295e3ef570
I20260809 23:14:12.453738 6623 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:14:12 dist-test-slave-7vjv krb5kdc[6473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:12.458292 6610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:12.448828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57228 (local address 127.2.76.254:39003)
0809 23:14:12.449010 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:12.449018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:12.449051 (+ 33us) server_negotiation.cc:407] Connection header received
0809 23:14:12.449326 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:12.449331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:12.449367 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:12.449469 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:12.450356 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:12.451687 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:12.452585 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:12.453135 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:12.453241 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:12.455341 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:12.455345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:12.455345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:12.455355 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:12.456860 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:12.457207 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:12.457210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:12.457210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:12.457308 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:12.457534 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:12.457535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:12.457536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:12.457761 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:12.457926 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:12.457985 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 23:14:12.458116 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":107,"threads_started":1}
I20260809 23:14:12.460484 6623 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:12.460903 6623 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:12.465794 6623 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b87a1d50bb54d21b0e5eee62e9b6f9f: Generated new TSK 0
I20260809 23:14:12.465947 6623 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:14:12.491189 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:39003
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:32823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:12.565456 6646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:12.565696 6646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:12.565747 6646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:12.566779 6646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:12.566840 6646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:12.566880 6646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:14:12.568580 6646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:32823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:39003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:12.568827 6646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:12.569067 6646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:12.574204 6652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:12.574220 6655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:12.574494 6653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:12.574677 6646 server_base.cc:1034] running on GCE node
I20260809 23:14:12.574904 6646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:12.576193 6646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:12.577368 6646 hybrid_clock.cc:648] HybridClock initialized: now 1786317252577356 us; error 27 us; skew 500 ppm
Aug 09 23:14:12 dist-test-slave-7vjv krb5kdc[6473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317252, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:12.580646 6646 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:14:12.581175 6646 webserver.cc:460] Webserver started at http://127.2.76.193:39291/ using document root <none> and password file <none>
I20260809 23:14:12.581415 6646 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:12.581488 6646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:12.581630 6646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:12.582528 6646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "d1793bb8e3cb4e5a9429c4eb75301b89"
format_stamp: "Formatted at 2026-08-09 23:14:12 on dist-test-slave-7vjv"
server_key: "fb614081f2a933d34f45ac3f16af559e"
server_key_iv: "9bc859adf4711282236ca1f0599010e5"
server_key_version: "encryptionkey@0"
I20260809 23:14:12.582872 6646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "d1793bb8e3cb4e5a9429c4eb75301b89"
format_stamp: "Formatted at 2026-08-09 23:14:12 on dist-test-slave-7vjv"
server_key: "fb614081f2a933d34f45ac3f16af559e"
server_key_iv: "9bc859adf4711282236ca1f0599010e5"
server_key_version: "encryptionkey@0"
I20260809 23:14:12.584214 6646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:14:12.584991 6670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:12.585189 6646 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:12.585283 6646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "d1793bb8e3cb4e5a9429c4eb75301b89"
format_stamp: "Formatted at 2026-08-09 23:14:12 on dist-test-slave-7vjv"
server_key: "fb614081f2a933d34f45ac3f16af559e"
server_key_iv: "9bc859adf4711282236ca1f0599010e5"
server_key_version: "encryptionkey@0"
I20260809 23:14:12.585366 6646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:12.592661 6646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:12.594596 6646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:12.594728 6646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:12.594913 6646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:12.595196 6646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:12.595244 6646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:12.595280 6646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:12.595305 6646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:12.600740 6646 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:39977
I20260809 23:14:12.600811 6871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:39977 every 8 connection(s)
I20260809 23:14:12.601097 6646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:14:12.606734 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 6646
I20260809 23:14:12.606844 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:14:12.607009 2355 external_mini_cluster.cc:1428] Setting key d14b6aabd88319f9656f86153c857fb4
Aug 09 23:14:12 dist-test-slave-7vjv krb5kdc[6473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317252, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:12.612607 6610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:12.601907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:60781 (local address 127.2.76.254:39003)
0809 23:14:12.601974 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:12.601976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:12.602241 (+ 265us) server_negotiation.cc:407] Connection header received
0809 23:14:12.602999 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:12.603000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:12.603016 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:12.603070 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:12.604388 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:12.604834 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:12.606399 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:12.606607 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:12.606675 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:12.609209 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:12.609211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:12.609211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:12.609220 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:12.610988 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:12.611439 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:12.611440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:12.611441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:12.611499 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:12.611760 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:12.611761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:12.611761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:12.611888 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:12.611927 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:12.612405 (+ 478us) server_negotiation.cc:299] Negotiation successful
0809 23:14:12.612498 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 23:14:12.612857 6875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:12.601868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:39003 (local address 127.2.76.193:60781)
0809 23:14:12.602105 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:12.602156 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:12.602859 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:12.603147 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:12.603153 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:12.603978 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:12.604288 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:12.604295 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:12.604923 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:12.604924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:12.606298 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:12.606302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:12.606701 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:12.606702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:12.606862 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:12.607439 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:12.607446 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:12.609088 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:12.611078 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:12.611084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:12.611086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:12.611356 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:12.611566 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:12.611566 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:12.611567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:12.611694 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:12.611982 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:12.611988 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:12.612235 (+ 247us) client_negotiation.cc:769] Sending connection context
0809 23:14:12.612365 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 23:14:12.612540 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":113,"threads_started":1}
I20260809 23:14:12.613358 6872 heartbeater.cc:347] Connected to a master server at 127.2.76.254:39003
I20260809 23:14:12.613461 6872 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:12.613651 6872 heartbeater.cc:511] Master 127.2.76.254:39003 requested a full tablet report, sending...
I20260809 23:14:12.614207 6547 ts_manager.cc:194] Registered new tserver with Master: d1793bb8e3cb4e5a9429c4eb75301b89 (127.2.76.193:39977)
I20260809 23:14:12.615082 6547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:60781
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:14:12.633316 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:39003
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:32823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:12.715229 6878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:12.715489 6878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:12.715528 6878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:12.717621 6878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:12.717679 6878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:12.717722 6878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:14:12.720013 6878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:32823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:39003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:12.720268 6878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:12.720544 6878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:12.725777 6888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:12.725803 6890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:12.725898 6892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:12.726167 6878 server_base.cc:1034] running on GCE node
I20260809 23:14:12.726382 6878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:12.726629 6878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:12.727799 6878 hybrid_clock.cc:648] HybridClock initialized: now 1786317252727777 us; error 30 us; skew 500 ppm
Aug 09 23:14:12 dist-test-slave-7vjv krb5kdc[6473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317252, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:12.731047 6878 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:14:12.731446 6878 webserver.cc:460] Webserver started at http://127.2.76.194:37689/ using document root <none> and password file <none>
I20260809 23:14:12.731678 6878 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:12.731753 6878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:12.731885 6878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:12.732834 6878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "cf6dd440e98f4304a41f4b01fed5609b"
format_stamp: "Formatted at 2026-08-09 23:14:12 on dist-test-slave-7vjv"
server_key: "8e12dcbfd8a601ed2e1448ccb271ec5a"
server_key_iv: "6e7e0d594ca041a3723c323119f76705"
server_key_version: "encryptionkey@0"
I20260809 23:14:12.733171 6878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "cf6dd440e98f4304a41f4b01fed5609b"
format_stamp: "Formatted at 2026-08-09 23:14:12 on dist-test-slave-7vjv"
server_key: "8e12dcbfd8a601ed2e1448ccb271ec5a"
server_key_iv: "6e7e0d594ca041a3723c323119f76705"
server_key_version: "encryptionkey@0"
I20260809 23:14:12.734500 6878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:14:12.735288 6903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:12.735469 6878 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:12.735569 6878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "cf6dd440e98f4304a41f4b01fed5609b"
format_stamp: "Formatted at 2026-08-09 23:14:12 on dist-test-slave-7vjv"
server_key: "8e12dcbfd8a601ed2e1448ccb271ec5a"
server_key_iv: "6e7e0d594ca041a3723c323119f76705"
server_key_version: "encryptionkey@0"
I20260809 23:14:12.735651 6878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:12.747953 6878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:12.750201 6878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:12.750332 6878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:12.750530 6878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:12.750814 6878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:12.750862 6878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:12.750896 6878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:12.750914 6878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:12.757133 6878 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:42709
I20260809 23:14:12.757179 7084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:42709 every 8 connection(s)
I20260809 23:14:12.757428 6878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:14:12.758951 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 6878
I20260809 23:14:12.759087 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:14:12.759287 2355 external_mini_cluster.cc:1428] Setting key a438f695f28c2bc7043e62e6985bc670
Aug 09 23:14:12 dist-test-slave-7vjv krb5kdc[6473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317252, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:12.767035 6610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:12.758101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:60263 (local address 127.2.76.254:39003)
0809 23:14:12.758168 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:12.758171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:12.758309 (+ 138us) server_negotiation.cc:407] Connection header received
0809 23:14:12.758699 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:12.758700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:12.758714 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:12.758786 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:12.759879 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:12.760720 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:12.761983 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:12.762179 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:12.762249 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:12.764175 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:12.764177 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:12.764177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:12.764188 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:12.765599 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:12.766059 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:12.766061 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:12.766061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:12.766124 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:12.766386 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:12.766387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:12.766387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:12.766507 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:12.766581 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:12.766846 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 23:14:12.766935 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 23:14:12.767220 7089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:12.758036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:39003 (local address 127.2.76.194:60263)
0809 23:14:12.758193 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:12.758228 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:12.758613 (+ 385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:12.758850 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:12.758854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:12.759532 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:12.759759 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:12.759765 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:12.760828 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:12.760829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:12.761898 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:12.761901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:12.762255 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:12.762257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:12.762367 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:12.762822 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:12.762830 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:12.764038 (+ 1208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:12.765695 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:12.765703 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:12.765704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:12.765970 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:12.766192 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:12.766193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:12.766193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:12.766309 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:12.766572 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:12.766580 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:12.766714 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 23:14:12.766820 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 23:14:12.766984 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":77,"threads_started":1}
I20260809 23:14:12.767779 7086 heartbeater.cc:347] Connected to a master server at 127.2.76.254:39003
I20260809 23:14:12.767887 7086 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:12.768105 7086 heartbeater.cc:511] Master 127.2.76.254:39003 requested a full tablet report, sending...
I20260809 23:14:12.768458 6547 ts_manager.cc:194] Registered new tserver with Master: cf6dd440e98f4304a41f4b01fed5609b (127.2.76.194:42709)
I20260809 23:14:12.768853 6547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:60263
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:14:12.789014 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:39003
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:32823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:12.873032 7097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:12.873194 7097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:12.873214 7097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:12.874272 7097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:12.874308 7097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:12.874351 7097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:14:12.875591 7097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:32823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:39003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:12.875805 7097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:12.876017 7097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:12.880264 7112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:12.880250 7114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:12.880299 7118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:12.880415 7097 server_base.cc:1034] running on GCE node
I20260809 23:14:12.880674 7097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:12.880841 7097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:12.881968 7097 hybrid_clock.cc:648] HybridClock initialized: now 1786317252881958 us; error 24 us; skew 500 ppm
Aug 09 23:14:12 dist-test-slave-7vjv krb5kdc[6473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317252, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:12.884313 7097 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:14:12.884732 7097 webserver.cc:460] Webserver started at http://127.2.76.195:38247/ using document root <none> and password file <none>
I20260809 23:14:12.884931 7097 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:12.884990 7097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:12.885123 7097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:12.885958 7097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "46313eb5f32742ffb6f02147bfca265d"
format_stamp: "Formatted at 2026-08-09 23:14:12 on dist-test-slave-7vjv"
server_key: "59d34bb0948db6a4d7e210974098ee83"
server_key_iv: "e2c1ecbfa3deafd450428f279737242a"
server_key_version: "encryptionkey@0"
I20260809 23:14:12.886272 7097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "46313eb5f32742ffb6f02147bfca265d"
format_stamp: "Formatted at 2026-08-09 23:14:12 on dist-test-slave-7vjv"
server_key: "59d34bb0948db6a4d7e210974098ee83"
server_key_iv: "e2c1ecbfa3deafd450428f279737242a"
server_key_version: "encryptionkey@0"
I20260809 23:14:12.887483 7097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:14:12.888191 7132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:12.888379 7097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:12.888453 7097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "46313eb5f32742ffb6f02147bfca265d"
format_stamp: "Formatted at 2026-08-09 23:14:12 on dist-test-slave-7vjv"
server_key: "59d34bb0948db6a4d7e210974098ee83"
server_key_iv: "e2c1ecbfa3deafd450428f279737242a"
server_key_version: "encryptionkey@0"
I20260809 23:14:12.888528 7097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:12.907109 7097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:12.908675 7097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:12.908807 7097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:12.908993 7097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:12.909243 7097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:12.909291 7097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:12.909325 7097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:12.909349 7097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:12.914494 7097 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:36593
I20260809 23:14:12.914582 7326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:36593 every 8 connection(s)
I20260809 23:14:12.914810 7097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:14:12 dist-test-slave-7vjv krb5kdc[6473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317252, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:12.922350 6610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:12.915433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:49877 (local address 127.2.76.254:39003)
0809 23:14:12.915489 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:12.915492 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:12.915659 (+ 167us) server_negotiation.cc:407] Connection header received
0809 23:14:12.916058 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:12.916060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:12.916071 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:12.916137 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:12.916893 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:12.917333 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:12.918178 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:12.918359 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:12.918426 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:12.920312 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:12.920314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:12.920314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:12.920320 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:12.921326 (+ 1006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:12.921637 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:12.921637 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:12.921638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:12.921683 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:12.921865 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:12.921866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:12.921866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:12.921950 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:12.921995 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:12.922195 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 23:14:12.922249 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 23:14:12.922425 7334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:12.915404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:39003 (local address 127.2.76.195:49877)
0809 23:14:12.915554 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:12.915583 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:12.915960 (+ 377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:12.916196 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:12.916199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:12.916650 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:12.916798 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:12.916802 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:12.917419 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:12.917420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:12.918089 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:12.918091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:12.918434 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:12.918436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:12.918545 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:12.918962 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:12.918969 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:12.920231 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:12.921388 (+ 1157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:12.921394 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:12.921395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:12.921579 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:12.921734 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:12.921735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:12.921735 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:12.921819 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:12.922010 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:12.922014 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:12.922101 (+ 87us) client_negotiation.cc:769] Sending connection context
0809 23:14:12.922168 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 23:14:12.922279 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260809 23:14:12.922863 7330 heartbeater.cc:347] Connected to a master server at 127.2.76.254:39003
I20260809 23:14:12.922957 7330 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:12.923107 7330 heartbeater.cc:511] Master 127.2.76.254:39003 requested a full tablet report, sending...
I20260809 23:14:12.923465 6547 ts_manager.cc:194] Registered new tserver with Master: 46313eb5f32742ffb6f02147bfca265d (127.2.76.195:36593)
I20260809 23:14:12.923784 6547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:49877
I20260809 23:14:12.923836 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 7097
I20260809 23:14:12.923930 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:14:12.924127 2355 external_mini_cluster.cc:1428] Setting key 73f9619abea79c8efdc83abd6ab2c4a9
I20260809 23:14:12.924842 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 23:14:12.936793 6610 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:14:12.936046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57244 (local address 127.2.76.254:39003)
0809 23:14:12.936150 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:12.936152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:12.936240 (+ 88us) server_negotiation.cc:407] Connection header received
0809 23:14:12.936308 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:12.936308 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:12.936316 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:12.936353 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:12.936752 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57244: BlockingRecv error: recv got EOF from 127.0.0.1:57244 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 23:14:12.937739 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 6646
I20260809 23:14:12.961131 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 6878
I20260809 23:14:12.970744 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 7097
I20260809 23:14:12.979245 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 6491
2026-08-09T23:14:12Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1927 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:14:13 dist-test-slave-7vjv krb5kdc[7351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:14:13 dist-test-slave-7vjv krb5kdc[7351](info): set up 2 sockets
Aug 09 23:14:13 dist-test-slave-7vjv krb5kdc[7351](info): commencing operation
krb5kdc: starting...
W20260809 23:14:14.127789 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:14:14 dist-test-slave-7vjv krb5kdc[7351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:14:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:14:14.232932 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:38239
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:38239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:14.305783 7368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:14.306031 7368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:14.306070 7368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:14.307261 7368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:14.307323 7368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:14.307346 7368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:14.307374 7368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:14.307401 7368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:14:14.308781 7368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:38239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:38239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:14.309020 7368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:14.309213 7368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:14.313463 7375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:14.313622 7368 server_base.cc:1034] running on GCE node
W20260809 23:14:14.313465 7376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:14.313443 7379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:14.313990 7368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:14.314261 7368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:14.315407 7368 hybrid_clock.cc:648] HybridClock initialized: now 1786317254315405 us; error 34 us; skew 500 ppm
Aug 09 23:14:14 dist-test-slave-7vjv krb5kdc[7351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317254, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:14.317575 7368 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:14:14.317925 7368 webserver.cc:460] Webserver started at http://127.2.76.254:42835/ using document root <none> and password file <none>
I20260809 23:14:14.318112 7368 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:14.318169 7368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:14.318293 7368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:14.319140 7368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "66cc1af5cd684f2fb93a883af784e4d2"
format_stamp: "Formatted at 2026-08-09 23:14:14 on dist-test-slave-7vjv"
server_key: "719ca1edaffbd98b3d1e953be161853e"
server_key_iv: "0e21ce7a1268b6c0b42792365c5d6a5b"
server_key_version: "encryptionkey@0"
I20260809 23:14:14.319442 7368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "66cc1af5cd684f2fb93a883af784e4d2"
format_stamp: "Formatted at 2026-08-09 23:14:14 on dist-test-slave-7vjv"
server_key: "719ca1edaffbd98b3d1e953be161853e"
server_key_iv: "0e21ce7a1268b6c0b42792365c5d6a5b"
server_key_version: "encryptionkey@0"
I20260809 23:14:14.320700 7368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:14:14.321298 7387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:14.321449 7368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:14.321538 7368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "66cc1af5cd684f2fb93a883af784e4d2"
format_stamp: "Formatted at 2026-08-09 23:14:14 on dist-test-slave-7vjv"
server_key: "719ca1edaffbd98b3d1e953be161853e"
server_key_iv: "0e21ce7a1268b6c0b42792365c5d6a5b"
server_key_version: "encryptionkey@0"
I20260809 23:14:14.321609 7368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:14.345439 7368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:14.347210 7368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:14.347357 7368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:14.351086 7368 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:38239
I20260809 23:14:14.351135 7477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:38239 every 8 connection(s)
I20260809 23:14:14.351457 7368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:14.352311 7478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:14.354226 7478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2: Bootstrap starting.
I20260809 23:14:14.354764 7478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:14.355033 7478 log.cc:824] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:14.355659 7478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2: No bootstrap required, opened a new log
I20260809 23:14:14.356959 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 7368
I20260809 23:14:14.357067 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:14.357225 2355 external_mini_cluster.cc:1428] Setting key 5bb68bc785d1f3a11734bf11cb4baf14
I20260809 23:14:14.357229 7478 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66cc1af5cd684f2fb93a883af784e4d2" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38239 } }
I20260809 23:14:14.357388 7478 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:14.357430 7478 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66cc1af5cd684f2fb93a883af784e4d2, State: Initialized, Role: FOLLOWER
I20260809 23:14:14.357563 7478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66cc1af5cd684f2fb93a883af784e4d2" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38239 } }
I20260809 23:14:14.357648 7478 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:14.357700 7478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:14.357752 7478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:14.358278 7478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66cc1af5cd684f2fb93a883af784e4d2" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38239 } }
I20260809 23:14:14.358376 7478 leader_election.cc:302] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66cc1af5cd684f2fb93a883af784e4d2; no voters:
I20260809 23:14:14.358534 7478 leader_election.cc:288] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:14.358587 7490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:14.358716 7490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [term 1 LEADER]: Becoming Leader. State: Replica: 66cc1af5cd684f2fb93a883af784e4d2, State: Running, Role: LEADER
I20260809 23:14:14.358768 7478 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:14.360160 7490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66cc1af5cd684f2fb93a883af784e4d2" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38239 } }
I20260809 23:14:14.360770 7491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66cc1af5cd684f2fb93a883af784e4d2. Latest consensus state: current_term: 1 leader_uuid: "66cc1af5cd684f2fb93a883af784e4d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66cc1af5cd684f2fb93a883af784e4d2" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38239 } } }
I20260809 23:14:14.360863 7493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66cc1af5cd684f2fb93a883af784e4d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66cc1af5cd684f2fb93a883af784e4d2" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38239 } } }
I20260809 23:14:14.361404 7491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:14.361428 7493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2 [sys.catalog]: This master's current role is: LEADER
W20260809 23:14:14.361487 7520 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:14:14.361529 7520 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:14:14.361646 7529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:14.361819 7529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:14.363179 7529 catalog_manager.cc:1347] Generated new cluster ID: d7e298c1d3ea4875b432d5aea17ceef7
I20260809 23:14:14.363236 7529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:14:14 dist-test-slave-7vjv krb5kdc[7351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:14.367627 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:14.357661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42120 (local address 127.2.76.254:38239)
0809 23:14:14.358146 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:14.358154 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:14.358184 (+ 30us) server_negotiation.cc:407] Connection header received
0809 23:14:14.358289 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:14.358295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:14.358343 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:14.358476 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:14.359261 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:14.360321 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:14.361280 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:14.361838 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:14.361949 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:14.364434 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:14.364437 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:14.364437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:14.364447 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:14.366079 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:14.366418 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:14.366421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:14.366421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:14.366721 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:14.366918 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:14.366918 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:14.366919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:14.367127 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:14.367288 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:14.367346 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 23:14:14.367463 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":384,"threads_started":1}
I20260809 23:14:14.368460 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:14:14.382057 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:14.378997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42130 (local address 127.2.76.254:38239)
0809 23:14:14.379036 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:14.379038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:14.379187 (+ 149us) server_negotiation.cc:407] Connection header received
0809 23:14:14.379260 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:14.379260 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:14.379276 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:14.379328 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:14.379710 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:14.380170 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:14.380801 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:14.380986 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:14.381054 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:14.381642 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:14.381643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:14.381643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:14.381821 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:14.381834 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:14.381839 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:14.381900 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:14.381967 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 23:14:14.381985 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:14:14.390725 7529 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:14.391312 7529 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:14.396051 7529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66cc1af5cd684f2fb93a883af784e4d2: Generated new TSK 0
I20260809 23:14:14.396250 7529 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:14:14.404749 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 7368
2026-08-09T23:14:14Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1434 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:14:14 dist-test-slave-7vjv krb5kdc[7550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:14:14 dist-test-slave-7vjv krb5kdc[7550](info): set up 2 sockets
Aug 09 23:14:14 dist-test-slave-7vjv krb5kdc[7550](info): commencing operation
krb5kdc: starting...
W20260809 23:14:15.969610 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.532s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:14:16 dist-test-slave-7vjv krb5kdc[7550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:14:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:14:16.118995 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:44199
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:38675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:44199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:16.252249 7568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:16.252599 7568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:16.252657 7568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:16.254405 7568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:16.254495 7568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:16.254556 7568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:16.254594 7568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:16.254653 7568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:14:16.256857 7568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:38675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:44199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:44199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:16.257311 7568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:16.257680 7568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:16.264842 7576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:16.264869 7575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:16.264909 7579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:16.265208 7568 server_base.cc:1034] running on GCE node
I20260809 23:14:16.265666 7568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:16.266173 7568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:16.267385 7568 hybrid_clock.cc:648] HybridClock initialized: now 1786317256267352 us; error 46 us; skew 500 ppm
Aug 09 23:14:16 dist-test-slave-7vjv krb5kdc[7550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317256, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:16.271301 7568 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:14:16.272040 7568 webserver.cc:460] Webserver started at http://127.2.76.254:41509/ using document root <none> and password file <none>
I20260809 23:14:16.272418 7568 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:16.272540 7568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:16.272774 7568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:16.274279 7568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "6116ebc06cb7484dadacdfb670c07e43"
format_stamp: "Formatted at 2026-08-09 23:14:16 on dist-test-slave-7vjv"
server_key: "15f42e3d5b5d94f52c003167b87f0e44"
server_key_iv: "f586dc1be97d7af1470f9a495b797bab"
server_key_version: "encryptionkey@0"
I20260809 23:14:16.274856 7568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "6116ebc06cb7484dadacdfb670c07e43"
format_stamp: "Formatted at 2026-08-09 23:14:16 on dist-test-slave-7vjv"
server_key: "15f42e3d5b5d94f52c003167b87f0e44"
server_key_iv: "f586dc1be97d7af1470f9a495b797bab"
server_key_version: "encryptionkey@0"
I20260809 23:14:16.277032 7568 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260809 23:14:16.278154 7590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:16.278424 7568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:16.278604 7568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "6116ebc06cb7484dadacdfb670c07e43"
format_stamp: "Formatted at 2026-08-09 23:14:16 on dist-test-slave-7vjv"
server_key: "15f42e3d5b5d94f52c003167b87f0e44"
server_key_iv: "f586dc1be97d7af1470f9a495b797bab"
server_key_version: "encryptionkey@0"
I20260809 23:14:16.278702 7568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:16.323714 7568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:16.326758 7568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:16.327014 7568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:16.333959 7568 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:44199
I20260809 23:14:16.334028 7681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:44199 every 8 connection(s)
I20260809 23:14:16.334599 7568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:16.335666 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 7568
I20260809 23:14:16.335865 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:16.336259 2355 external_mini_cluster.cc:1428] Setting key 3fde04177177bedf062a1b4d9255246e
I20260809 23:14:16.337102 7683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:16.343870 7683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43: Bootstrap starting.
I20260809 23:14:16.345204 7683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:16.345810 7683 log.cc:824] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:16.347124 7683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43: No bootstrap required, opened a new log
I20260809 23:14:16.351445 7683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6116ebc06cb7484dadacdfb670c07e43" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44199 } }
I20260809 23:14:16.351699 7683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:16.351778 7683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6116ebc06cb7484dadacdfb670c07e43, State: Initialized, Role: FOLLOWER
I20260809 23:14:16.351997 7683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6116ebc06cb7484dadacdfb670c07e43" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44199 } }
I20260809 23:14:16.352159 7683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:16.352252 7683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:16.352373 7683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:16.353441 7683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6116ebc06cb7484dadacdfb670c07e43" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44199 } }
I20260809 23:14:16.353617 7683 leader_election.cc:302] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6116ebc06cb7484dadacdfb670c07e43; no voters:
I20260809 23:14:16.353902 7683 leader_election.cc:288] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:16.354171 7691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:16.354363 7683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:16.354532 7691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [term 1 LEADER]: Becoming Leader. State: Replica: 6116ebc06cb7484dadacdfb670c07e43, State: Running, Role: LEADER
I20260809 23:14:16.354782 7691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6116ebc06cb7484dadacdfb670c07e43" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44199 } }
I20260809 23:14:16.355440 7691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6116ebc06cb7484dadacdfb670c07e43. Latest consensus state: current_term: 1 leader_uuid: "6116ebc06cb7484dadacdfb670c07e43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6116ebc06cb7484dadacdfb670c07e43" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44199 } } }
I20260809 23:14:16.355741 7691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:16.355762 7689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6116ebc06cb7484dadacdfb670c07e43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6116ebc06cb7484dadacdfb670c07e43" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44199 } } }
I20260809 23:14:16.355930 7689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:16.356365 7702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 23:14:16 dist-test-slave-7vjv krb5kdc[7550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:16.357712 7702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:16.360720 7702 catalog_manager.cc:1347] Generated new cluster ID: b2e0bafb50ec40b3a307542fdd9d7e15
I20260809 23:14:16.360791 7702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:14:16.364970 7687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:16.337412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46486 (local address 127.2.76.254:44199)
0809 23:14:16.337776 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:16.337794 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:16.344364 (+ 6570us) server_negotiation.cc:407] Connection header received
0809 23:14:16.345579 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:16.345597 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:16.345740 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:16.346069 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:16.347597 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:16.349746 (+ 2149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:16.352340 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:16.353240 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:16.353474 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:16.358322 (+ 4848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:16.358337 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:16.358337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:16.358360 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:16.362029 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:16.362663 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:16.362667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:16.362667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:16.362850 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:16.363219 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:16.363220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:16.363220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:16.363769 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:16.364065 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:16.364244 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 23:14:16.364558 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":137,"threads_started":1}
I20260809 23:14:16.370704 7702 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:16.371642 7702 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:16.386251 7702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6116ebc06cb7484dadacdfb670c07e43: Generated new TSK 0
I20260809 23:14:16.386529 7702 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:14:16.427062 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:44199
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:38675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:16.557579 7722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:16.557931 7722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:16.557979 7722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:16.559818 7722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:16.559890 7722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:16.559935 7722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:14:16.562446 7722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:38675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:44199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:16.562784 7722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:16.563124 7722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:16.571851 7731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:16.571851 7733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:16.571954 7730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:16.572880 7722 server_base.cc:1034] running on GCE node
I20260809 23:14:16.573232 7722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:16.573721 7722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:16.574972 7722 hybrid_clock.cc:648] HybridClock initialized: now 1786317256574934 us; error 67 us; skew 500 ppm
Aug 09 23:14:16 dist-test-slave-7vjv krb5kdc[7550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317256, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:16.578795 7722 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:14:16.579627 7722 webserver.cc:460] Webserver started at http://127.2.76.193:40313/ using document root <none> and password file <none>
I20260809 23:14:16.580000 7722 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:16.580158 7722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:16.580451 7722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:16.582083 7722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "8b5feae9799a4335a669b130be2a0228"
format_stamp: "Formatted at 2026-08-09 23:14:16 on dist-test-slave-7vjv"
server_key: "5c108e443b727b040516b0bfaa5fd0cf"
server_key_iv: "4fe43b938356a5447e91539ac53b60a2"
server_key_version: "encryptionkey@0"
I20260809 23:14:16.582711 7722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "8b5feae9799a4335a669b130be2a0228"
format_stamp: "Formatted at 2026-08-09 23:14:16 on dist-test-slave-7vjv"
server_key: "5c108e443b727b040516b0bfaa5fd0cf"
server_key_iv: "4fe43b938356a5447e91539ac53b60a2"
server_key_version: "encryptionkey@0"
I20260809 23:14:16.585407 7722 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.005s sys 0.000s
I20260809 23:14:16.586603 7749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:16.586860 7722 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:16.587052 7722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "8b5feae9799a4335a669b130be2a0228"
format_stamp: "Formatted at 2026-08-09 23:14:16 on dist-test-slave-7vjv"
server_key: "5c108e443b727b040516b0bfaa5fd0cf"
server_key_iv: "4fe43b938356a5447e91539ac53b60a2"
server_key_version: "encryptionkey@0"
I20260809 23:14:16.587189 7722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:16.598392 7722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:16.601588 7722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:16.601797 7722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:16.602142 7722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:16.602547 7722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:16.602624 7722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:16.602703 7722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:16.602773 7722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:16.612749 7722 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:39437
I20260809 23:14:16.612807 7949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:39437 every 8 connection(s)
I20260809 23:14:16.613329 7722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:14:16.614461 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 7722
I20260809 23:14:16.614627 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:14:16.614887 2355 external_mini_cluster.cc:1428] Setting key 763aa46e1158512e2f3c9a958075fae5
Aug 09 23:14:16 dist-test-slave-7vjv krb5kdc[7550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317256, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:16.627981 7687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:16.614381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:53289 (local address 127.2.76.254:44199)
0809 23:14:16.614460 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:16.614462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:16.614745 (+ 283us) server_negotiation.cc:407] Connection header received
0809 23:14:16.615660 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:16.615661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:16.615673 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:16.615820 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:16.617397 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:16.618097 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:16.619423 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:16.619764 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:16.619929 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:16.623092 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:16.623094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:16.623094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:16.623103 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:16.625741 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:16.626484 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:16.626485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:16.626485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:16.626572 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:16.626946 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:16.626947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:16.626947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:16.627110 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:16.627243 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:16.627682 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 23:14:16.627817 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:14:16.628381 7956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:16.614344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:44199 (local address 127.2.76.193:53289)
0809 23:14:16.614579 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:16.614649 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:16.615513 (+ 864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:16.615987 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:16.616001 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:16.616977 (+ 976us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:16.617253 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:16.617260 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:16.618252 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:16.618253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:16.619263 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:16.619265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:16.619935 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:16.619941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:16.620175 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:16.620915 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:16.620924 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:16.622896 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:16.625926 (+ 3030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:16.625942 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:16.625945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:16.626326 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:16.626679 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:16.626680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:16.626680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:16.626845 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:16.627227 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:16.627235 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:16.627438 (+ 203us) client_negotiation.cc:769] Sending connection context
0809 23:14:16.627638 (+ 200us) client_negotiation.cc:240] Negotiation successful
0809 23:14:16.627915 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":145,"threads_started":1}
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
I20260809 23:14:16.629395 7950 heartbeater.cc:347] Connected to a master server at 127.2.76.254:44199
I20260809 23:14:16.629585 7950 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:16.630010 7950 heartbeater.cc:511] Master 127.2.76.254:44199 requested a full tablet report, sending...
I20260809 23:14:16.630834 7622 ts_manager.cc:194] Registered new tserver with Master: 8b5feae9799a4335a669b130be2a0228 (127.2.76.193:39437)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
I20260809 23:14:16.631821 7622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:53289
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:14:16.651226 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:44199
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:38675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:16.785130 7960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:16.785398 7960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:16.785465 7960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:16.787101 7960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:16.787159 7960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:16.787242 7960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:14:16.789445 7960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:38675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:44199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:16.789754 7960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:16.790026 7960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:16.797222 7971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:16.797447 7967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:16.797276 7966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:16.797981 7960 server_base.cc:1034] running on GCE node
I20260809 23:14:16.798432 7960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:16.798923 7960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:16.800184 7960 hybrid_clock.cc:648] HybridClock initialized: now 1786317256800144 us; error 55 us; skew 500 ppm
Aug 09 23:14:16 dist-test-slave-7vjv krb5kdc[7550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317256, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:16.805034 7960 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:14:16.805783 7960 webserver.cc:460] Webserver started at http://127.2.76.194:34793/ using document root <none> and password file <none>
I20260809 23:14:16.806165 7960 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:16.806313 7960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:16.806535 7960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:16.808318 7960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "327d6572dac74350b10f18adb9af6615"
format_stamp: "Formatted at 2026-08-09 23:14:16 on dist-test-slave-7vjv"
server_key: "acc14b0d8e186995754889c141527181"
server_key_iv: "9465f5ebefb05c168323f4172dd952f6"
server_key_version: "encryptionkey@0"
I20260809 23:14:16.808858 7960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "327d6572dac74350b10f18adb9af6615"
format_stamp: "Formatted at 2026-08-09 23:14:16 on dist-test-slave-7vjv"
server_key: "acc14b0d8e186995754889c141527181"
server_key_iv: "9465f5ebefb05c168323f4172dd952f6"
server_key_version: "encryptionkey@0"
I20260809 23:14:16.811105 7960 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:16.812469 7990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:16.812870 7960 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:16.813025 7960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "327d6572dac74350b10f18adb9af6615"
format_stamp: "Formatted at 2026-08-09 23:14:16 on dist-test-slave-7vjv"
server_key: "acc14b0d8e186995754889c141527181"
server_key_iv: "9465f5ebefb05c168323f4172dd952f6"
server_key_version: "encryptionkey@0"
I20260809 23:14:16.813154 7960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:16.847275 7960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:16.850829 7960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:16.851046 7960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:16.851373 7960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:16.851858 7960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:16.851938 7960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:16.852005 7960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:16.852056 7960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:16.864681 7960 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:35003
I20260809 23:14:16.864930 8189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:35003 every 8 connection(s)
I20260809 23:14:16.865533 7960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:14:16.868934 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 7960
I20260809 23:14:16.869030 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:14:16.869352 2355 external_mini_cluster.cc:1428] Setting key 86eb6127a43243bf5f62a3eb6b785bab
Aug 09 23:14:16 dist-test-slave-7vjv krb5kdc[7550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317256, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:16.878890 8195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:16.866639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:44199 (local address 127.2.76.194:37931)
0809 23:14:16.866913 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:16.866987 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:16.867847 (+ 860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:16.868219 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:16.868229 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:16.869296 (+ 1067us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:16.869865 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:16.869877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:16.871362 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:16.871363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:16.872882 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:16.872890 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:16.873789 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:16.873790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:16.873931 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:16.874387 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:16.874397 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:16.875818 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:16.877644 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:16.877650 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:16.877652 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:16.877846 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:16.878064 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:16.878064 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:16.878064 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:16.878153 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:16.878453 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:16.878457 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:16.878566 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 23:14:16.878636 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 23:14:16.878736 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":125,"threads_started":1}
I20260809 23:14:16.878913 7687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:16.866633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:37931 (local address 127.2.76.254:44199)
0809 23:14:16.866698 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:16.866700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:16.867115 (+ 415us) server_negotiation.cc:407] Connection header received
0809 23:14:16.867945 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:16.867946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:16.867964 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:16.868033 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:16.870039 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:16.871174 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:16.873019 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:16.873641 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:16.873784 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:16.875905 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:16.875906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:16.875906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:16.875914 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:16.877552 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:16.877921 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:16.877922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:16.877922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:16.877988 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:16.878216 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:16.878217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:16.878217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:16.878379 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:16.878452 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:16.878688 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 23:14:16.878772 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 23:14:16.879431 8191 heartbeater.cc:347] Connected to a master server at 127.2.76.254:44199
I20260809 23:14:16.879542 8191 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:16.879753 8191 heartbeater.cc:511] Master 127.2.76.254:44199 requested a full tablet report, sending...
I20260809 23:14:16.880146 7622 ts_manager.cc:194] Registered new tserver with Master: 327d6572dac74350b10f18adb9af6615 (127.2.76.194:35003)
I20260809 23:14:16.880517 7622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:37931
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:14:16.896492 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:44199
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:38675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:16.968521 8200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:16.968688 8200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:16.968706 8200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:16.969657 8200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:16.969691 8200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:16.969734 8200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:14:16.970939 8200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:38675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:44199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:16.971161 8200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:16.971360 8200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:16.975872 8211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:16.975872 8217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:16.976050 8200 server_base.cc:1034] running on GCE node
W20260809 23:14:16.975888 8213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:16.976317 8200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:16.976542 8200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:16.977692 8200 hybrid_clock.cc:648] HybridClock initialized: now 1786317256977684 us; error 28 us; skew 500 ppm
Aug 09 23:14:16 dist-test-slave-7vjv krb5kdc[7550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317256, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:16.979889 8200 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:14:16.980333 8200 webserver.cc:460] Webserver started at http://127.2.76.195:34605/ using document root <none> and password file <none>
I20260809 23:14:16.980552 8200 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:16.980616 8200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:16.980782 8200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:16.981730 8200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "982988fab1b54f468ece1e28ef03afcf"
format_stamp: "Formatted at 2026-08-09 23:14:16 on dist-test-slave-7vjv"
server_key: "ee6b49df54c2e077f261f977634e1f6a"
server_key_iv: "a245cd087fe087a840cd58d528e06f8e"
server_key_version: "encryptionkey@0"
I20260809 23:14:16.982053 8200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "982988fab1b54f468ece1e28ef03afcf"
format_stamp: "Formatted at 2026-08-09 23:14:16 on dist-test-slave-7vjv"
server_key: "ee6b49df54c2e077f261f977634e1f6a"
server_key_iv: "a245cd087fe087a840cd58d528e06f8e"
server_key_version: "encryptionkey@0"
I20260809 23:14:16.983317 8200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:14:16.984031 8225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:16.984213 8200 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:16.984318 8200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "982988fab1b54f468ece1e28ef03afcf"
format_stamp: "Formatted at 2026-08-09 23:14:16 on dist-test-slave-7vjv"
server_key: "ee6b49df54c2e077f261f977634e1f6a"
server_key_iv: "a245cd087fe087a840cd58d528e06f8e"
server_key_version: "encryptionkey@0"
I20260809 23:14:16.984390 8200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:17.005489 8200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:17.007447 8200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:17.007611 8200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:17.007818 8200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:17.008131 8200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:17.008189 8200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:17.008227 8200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:17.008250 8200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:17.013911 8200 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:39475
I20260809 23:14:17.013947 8412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:39475 every 8 connection(s)
I20260809 23:14:17.014230 8200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:14:17 dist-test-slave-7vjv krb5kdc[7550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317256, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:17.020605 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 8200
I20260809 23:14:17.020713 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:14:17.020882 2355 external_mini_cluster.cc:1428] Setting key c44163f57ee8ca5dd84bd35d49643540
I20260809 23:14:17.023018 7687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:17.014915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:59181 (local address 127.2.76.254:44199)
0809 23:14:17.014963 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:17.014965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:17.015150 (+ 185us) server_negotiation.cc:407] Connection header received
0809 23:14:17.015559 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:17.015559 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:17.015570 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:17.015617 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:17.016459 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:17.016943 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:17.017791 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:17.017944 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:17.017986 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:17.020028 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:17.020029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:17.020029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:17.020036 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:17.021563 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:17.022019 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:17.022020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:17.022020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:17.022079 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:17.022329 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:17.022329 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:17.022330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:17.022482 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:17.022556 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:17.022829 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 23:14:17.022919 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:14:17.023149 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:17.014893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:44199 (local address 127.2.76.195:59181)
0809 23:14:17.015046 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:17.015077 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:17.015463 (+ 386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:17.015671 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:17.015673 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:17.016202 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:17.016367 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:17.016371 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:17.017016 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:17.017017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:17.017725 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:17.017726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:17.018007 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:17.018008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:17.018096 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:17.018544 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:17.018552 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:17.019947 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:17.021668 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:17.021675 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:17.021677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:17.021923 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:17.022142 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:17.022143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:17.022143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:17.022258 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:17.022561 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:17.022568 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:17.022694 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 23:14:17.022793 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 23:14:17.022940 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260809 23:14:17.023600 8415 heartbeater.cc:347] Connected to a master server at 127.2.76.254:44199
I20260809 23:14:17.023676 8415 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:17.023871 8415 heartbeater.cc:511] Master 127.2.76.254:44199 requested a full tablet report, sending...
I20260809 23:14:17.024267 7622 ts_manager.cc:194] Registered new tserver with Master: 982988fab1b54f468ece1e28ef03afcf (127.2.76.195:39475)
I20260809 23:14:17.024621 7622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:59181
I20260809 23:14:17.032523 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 23:14:17 dist-test-slave-7vjv krb5kdc[7550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317257, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 23:14:17 dist-test-slave-7vjv krb5kdc[7550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317257, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:17.053582 7687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:17.047687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46488 (local address 127.2.76.254:44199)
0809 23:14:17.047761 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:17.047763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:17.047833 (+ 70us) server_negotiation.cc:407] Connection header received
0809 23:14:17.047872 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:17.047872 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:17.047880 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:17.047926 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:17.048433 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:17.048816 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:17.049518 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:17.049682 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:17.049728 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:17.051672 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:17.051673 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:17.051674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:17.051679 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:17.052729 (+ 1050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:17.053012 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:17.053012 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:17.053012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:17.053061 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:17.053197 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:17.053197 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:17.053197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:17.053307 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:17.053375 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:17.053444 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 23:14:17.053496 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 23:14:17.053817 7622 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46488
I20260809 23:14:17.054365 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 7722
I20260809 23:14:17.084671 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 7960
I20260809 23:14:17.094219 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 8200
I20260809 23:14:17.102041 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 7568
2026-08-09T23:14:17Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2687 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:14:17 dist-test-slave-7vjv krb5kdc[8435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:14:17 dist-test-slave-7vjv krb5kdc[8435](info): set up 2 sockets
Aug 09 23:14:17 dist-test-slave-7vjv krb5kdc[8435](info): commencing operation
krb5kdc: starting...
W20260809 23:14:18.230196 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:14:18 dist-test-slave-7vjv krb5kdc[8435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:14:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:14:18.334861 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:37987
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:37987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:18.409348 8470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:18.409523 8470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:18.409543 8470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:18.410542 8470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:18.410578 8470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:18.410591 8470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:18.410624 8470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:18.410650 8470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:14:18.411964 8470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:37987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:37987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:18.412225 8470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:18.412431 8470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:18.416888 8478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:18.416889 8479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:18.416888 8481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:18.417094 8470 server_base.cc:1034] running on GCE node
I20260809 23:14:18.417322 8470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:18.417573 8470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:18.418720 8470 hybrid_clock.cc:648] HybridClock initialized: now 1786317258418705 us; error 28 us; skew 500 ppm
Aug 09 23:14:18 dist-test-slave-7vjv krb5kdc[8435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317258, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:18.423929 8470 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:14:18.424371 8470 webserver.cc:460] Webserver started at http://127.2.76.254:42311/ using document root <none> and password file <none>
I20260809 23:14:18.424579 8470 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:18.424638 8470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:18.424770 8470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:18.425709 8470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "a0265997a4e34813a5a5ebd953d8a63f"
format_stamp: "Formatted at 2026-08-09 23:14:18 on dist-test-slave-7vjv"
server_key: "eaafa1a5e5a4afd7d76b430b8d7a3c1b"
server_key_iv: "a3cbdd97c8d34a3e99ca96dd9a0c9a9d"
server_key_version: "encryptionkey@0"
I20260809 23:14:18.426105 8470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "a0265997a4e34813a5a5ebd953d8a63f"
format_stamp: "Formatted at 2026-08-09 23:14:18 on dist-test-slave-7vjv"
server_key: "eaafa1a5e5a4afd7d76b430b8d7a3c1b"
server_key_iv: "a3cbdd97c8d34a3e99ca96dd9a0c9a9d"
server_key_version: "encryptionkey@0"
I20260809 23:14:18.427295 8470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:14:18.427923 8492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:18.428078 8470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:18.428211 8470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "a0265997a4e34813a5a5ebd953d8a63f"
format_stamp: "Formatted at 2026-08-09 23:14:18 on dist-test-slave-7vjv"
server_key: "eaafa1a5e5a4afd7d76b430b8d7a3c1b"
server_key_iv: "a3cbdd97c8d34a3e99ca96dd9a0c9a9d"
server_key_version: "encryptionkey@0"
I20260809 23:14:18.428287 8470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:18.441638 8470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:18.443153 8470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:18.443298 8470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:18.447232 8470 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:37987
I20260809 23:14:18.447266 8573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:37987 every 8 connection(s)
I20260809 23:14:18.447738 8470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:18.448370 8574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:18.449626 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 8470
I20260809 23:14:18.449733 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:18.449904 2355 external_mini_cluster.cc:1428] Setting key c0858b8fcf8e85fdfd416921a7501631
I20260809 23:14:18.450716 8574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f: Bootstrap starting.
I20260809 23:14:18.451304 8574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:18.451632 8574 log.cc:824] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:18.452381 8574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f: No bootstrap required, opened a new log
I20260809 23:14:18.454449 8574 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0265997a4e34813a5a5ebd953d8a63f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37987 } }
I20260809 23:14:18.454605 8574 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:18.454644 8574 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0265997a4e34813a5a5ebd953d8a63f, State: Initialized, Role: FOLLOWER
I20260809 23:14:18.454767 8574 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0265997a4e34813a5a5ebd953d8a63f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37987 } }
I20260809 23:14:18.454849 8574 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:18.454888 8574 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:18.454947 8574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:18.455416 8574 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0265997a4e34813a5a5ebd953d8a63f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37987 } }
I20260809 23:14:18.455528 8574 leader_election.cc:302] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0265997a4e34813a5a5ebd953d8a63f; no voters:
I20260809 23:14:18.455677 8574 leader_election.cc:288] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:18.455703 8579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:18.455870 8579 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [term 1 LEADER]: Becoming Leader. State: Replica: a0265997a4e34813a5a5ebd953d8a63f, State: Running, Role: LEADER
I20260809 23:14:18.455909 8574 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:18.456137 8579 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0265997a4e34813a5a5ebd953d8a63f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37987 } }
Aug 09 23:14:18 dist-test-slave-7vjv krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:18.457482 8580 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0265997a4e34813a5a5ebd953d8a63f. Latest consensus state: current_term: 1 leader_uuid: "a0265997a4e34813a5a5ebd953d8a63f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0265997a4e34813a5a5ebd953d8a63f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37987 } } }
I20260809 23:14:18.457213 8581 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0265997a4e34813a5a5ebd953d8a63f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0265997a4e34813a5a5ebd953d8a63f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37987 } } }
I20260809 23:14:18.457769 8581 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [sys.catalog]: This master's current role is: LEADER
W20260809 23:14:18.457937 8600 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:14:18.457993 8600 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:14:18.458029 8580 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:18.458114 8609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:18.458276 8609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:18.459841 8609 catalog_manager.cc:1347] Generated new cluster ID: 89c7606edd06483f83eb4ccec4b58514
I20260809 23:14:18.459895 8609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:14:18.460645 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:18.450390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40036 (local address 127.2.76.254:37987)
0809 23:14:18.450546 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:18.450554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:18.450584 (+ 30us) server_negotiation.cc:407] Connection header received
0809 23:14:18.451010 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:18.451015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:18.451056 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:18.451155 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:18.451897 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.452818 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.453791 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.454162 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.454243 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:18.456994 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:18.456999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:18.456999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:18.457019 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:18.458938 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:18.459400 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:18.459403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:18.459404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:18.459499 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:18.459796 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:18.459797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:18.459797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:18.460059 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:18.460222 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:18.460348 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 23:14:18.460472 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260809 23:14:18.466006 8609 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:18.466645 8609 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:18.471177 8609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0265997a4e34813a5a5ebd953d8a63f: Generated new TSK 0
I20260809 23:14:18.471340 8609 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:14:18.500937 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37987
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:18.575495 8615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:18.575709 8615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:18.575744 8615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:18.576817 8615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:18.576881 8615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:18.576920 8615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:14:18.578307 8615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:18.578533 8615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:18.578770 8615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:18.583204 8645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:18.583216 8641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:18.583288 8615 server_base.cc:1034] running on GCE node
W20260809 23:14:18.583216 8624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:18.583635 8615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:18.583820 8615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:18.584950 8615 hybrid_clock.cc:648] HybridClock initialized: now 1786317258584918 us; error 46 us; skew 500 ppm
Aug 09 23:14:18 dist-test-slave-7vjv krb5kdc[8435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317258, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:18.587317 8615 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:14:18.587707 8615 webserver.cc:460] Webserver started at http://127.2.76.193:33075/ using document root <none> and password file <none>
I20260809 23:14:18.587898 8615 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:18.587935 8615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:18.588071 8615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:18.588973 8615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "1e9fd48396434ef8b3e370174b924b9c"
format_stamp: "Formatted at 2026-08-09 23:14:18 on dist-test-slave-7vjv"
server_key: "91c9e62fb8d50e1efa4b7cb0af96f265"
server_key_iv: "d662405414cf54475fc2133b6bf05db9"
server_key_version: "encryptionkey@0"
I20260809 23:14:18.589375 8615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "1e9fd48396434ef8b3e370174b924b9c"
format_stamp: "Formatted at 2026-08-09 23:14:18 on dist-test-slave-7vjv"
server_key: "91c9e62fb8d50e1efa4b7cb0af96f265"
server_key_iv: "d662405414cf54475fc2133b6bf05db9"
server_key_version: "encryptionkey@0"
I20260809 23:14:18.590653 8615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:14:18.591277 8662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:18.591454 8615 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:14:18.591549 8615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "1e9fd48396434ef8b3e370174b924b9c"
format_stamp: "Formatted at 2026-08-09 23:14:18 on dist-test-slave-7vjv"
server_key: "91c9e62fb8d50e1efa4b7cb0af96f265"
server_key_iv: "d662405414cf54475fc2133b6bf05db9"
server_key_version: "encryptionkey@0"
I20260809 23:14:18.591614 8615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:18.598390 8615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:18.600229 8615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:18.600394 8615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:18.600613 8615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:18.600967 8615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:18.601006 8615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:18.601028 8615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:18.601094 8615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:18.607080 8615 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:40955
I20260809 23:14:18.607152 8864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:40955 every 8 connection(s)
I20260809 23:14:18.607434 8615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:14:18 dist-test-slave-7vjv krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317258, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:18.615370 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 8615
I20260809 23:14:18.615477 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:14:18.615650 2355 external_mini_cluster.cc:1428] Setting key bbe3cc0592ff2434d061569a85bcd84f
I20260809 23:14:18.615782 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:18.608183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:39681 (local address 127.2.76.254:37987)
0809 23:14:18.608241 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:18.608243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:18.608412 (+ 169us) server_negotiation.cc:407] Connection header received
0809 23:14:18.609213 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:18.609215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:18.609227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:18.609283 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:18.609833 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.610288 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.611133 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.611338 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.611407 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:18.613264 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:18.613266 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:18.613266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:18.613273 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:18.614487 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:18.614854 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:18.614855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:18.614856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:18.614910 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:18.615142 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:18.615143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:18.615144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:18.615269 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:18.615331 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:18.615606 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 23:14:18.615682 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 23:14:18.615958 8873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:18.608159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37987 (local address 127.2.76.193:39681)
0809 23:14:18.608309 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:18.608346 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:18.609116 (+ 770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:18.609340 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:18.609344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:18.609567 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:18.609734 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:18.609738 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.610376 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.610377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:18.611052 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:18.611055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.611415 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.611416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:18.611534 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:18.611908 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:18.611915 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:18.613164 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:18.614580 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:18.614587 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:18.614589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:18.614784 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:18.614974 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:18.614975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:18.614975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:18.615077 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:18.615335 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:18.615341 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:18.615486 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 23:14:18.615584 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 23:14:18.615731 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260809 23:14:18.616595 8865 heartbeater.cc:347] Connected to a master server at 127.2.76.254:37987
I20260809 23:14:18.616665 8865 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:18.616818 8865 heartbeater.cc:511] Master 127.2.76.254:37987 requested a full tablet report, sending...
I20260809 23:14:18.617327 8524 ts_manager.cc:194] Registered new tserver with Master: 1e9fd48396434ef8b3e370174b924b9c (127.2.76.193:40955)
I20260809 23:14:18.618111 8524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:39681
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:14:18.638795 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37987
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:18.713562 8879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:18.713768 8879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:18.713802 8879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:18.714820 8879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:18.714876 8879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:18.714917 8879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:14:18.716274 8879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:18.716504 8879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:18.716703 8879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:14:18.721093 8879 server_base.cc:1034] running on GCE node
W20260809 23:14:18.721099 8897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:18.721102 8895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:18.721200 8902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:18.721555 8879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:18.721781 8879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:18.722911 8879 hybrid_clock.cc:648] HybridClock initialized: now 1786317258722905 us; error 29 us; skew 500 ppm
Aug 09 23:14:18 dist-test-slave-7vjv krb5kdc[8435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317258, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:18.725271 8879 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:14:18.725657 8879 webserver.cc:460] Webserver started at http://127.2.76.194:36271/ using document root <none> and password file <none>
I20260809 23:14:18.725883 8879 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:18.725922 8879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:18.726065 8879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:18.726910 8879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "73af9c733e564756bf3b434e68d48ce0"
format_stamp: "Formatted at 2026-08-09 23:14:18 on dist-test-slave-7vjv"
server_key: "f956dfe49e6f30cf8d460290c4346030"
server_key_iv: "2295703d4c394e7964ba5c381359ddca"
server_key_version: "encryptionkey@0"
I20260809 23:14:18.727231 8879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "73af9c733e564756bf3b434e68d48ce0"
format_stamp: "Formatted at 2026-08-09 23:14:18 on dist-test-slave-7vjv"
server_key: "f956dfe49e6f30cf8d460290c4346030"
server_key_iv: "2295703d4c394e7964ba5c381359ddca"
server_key_version: "encryptionkey@0"
I20260809 23:14:18.728462 8879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:14:18.729063 8915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:18.729206 8879 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:18.729295 8879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "73af9c733e564756bf3b434e68d48ce0"
format_stamp: "Formatted at 2026-08-09 23:14:18 on dist-test-slave-7vjv"
server_key: "f956dfe49e6f30cf8d460290c4346030"
server_key_iv: "2295703d4c394e7964ba5c381359ddca"
server_key_version: "encryptionkey@0"
I20260809 23:14:18.729362 8879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:18.737372 8879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:18.739300 8879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:18.739466 8879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:18.739681 8879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:18.739954 8879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:18.740020 8879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:18.740056 8879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:18.740082 8879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:18.745519 8879 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:38545
I20260809 23:14:18.745572 9133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:38545 every 8 connection(s)
I20260809 23:14:18.745814 8879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:14:18 dist-test-slave-7vjv krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317258, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:18.752861 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 8879
I20260809 23:14:18.752974 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:14:18.753165 2355 external_mini_cluster.cc:1428] Setting key d37cf5ceb4451ae5a76c28baee1e4a1a
I20260809 23:14:18.753342 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:18.746384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:34145 (local address 127.2.76.254:37987)
0809 23:14:18.746426 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:18.746427 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:18.746588 (+ 161us) server_negotiation.cc:407] Connection header received
0809 23:14:18.747305 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:18.747306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:18.747315 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:18.747361 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:18.747821 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.748244 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.749026 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.749179 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.749221 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:18.750858 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:18.750860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:18.750861 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:18.750867 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:18.752055 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:18.752440 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:18.752441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:18.752441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:18.752492 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:18.752706 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:18.752707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:18.752707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:18.752843 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:18.752914 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:18.753173 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 23:14:18.753251 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:14:18.753521 9137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:18.746341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37987 (local address 127.2.76.194:34145)
0809 23:14:18.746487 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:18.746527 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:18.747230 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:18.747420 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:18.747422 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:18.747622 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:18.747761 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:18.747765 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.748311 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.748312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:18.748961 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:18.748963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.749242 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.749242 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:18.749322 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:18.749656 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:18.749663 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:18.750786 (+ 1123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:18.752173 (+ 1387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:18.752179 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:18.752180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:18.752353 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:18.752554 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:18.752554 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:18.752555 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:18.752639 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:18.752908 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:18.752912 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:18.753068 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 23:14:18.753144 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 23:14:18.753287 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260809 23:14:18.754025 9134 heartbeater.cc:347] Connected to a master server at 127.2.76.254:37987
I20260809 23:14:18.754112 9134 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:18.754266 9134 heartbeater.cc:511] Master 127.2.76.254:37987 requested a full tablet report, sending...
I20260809 23:14:18.754634 8524 ts_manager.cc:194] Registered new tserver with Master: 73af9c733e564756bf3b434e68d48ce0 (127.2.76.194:38545)
I20260809 23:14:18.755084 8524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:34145
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:14:18.776222 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37987
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:18.849304 9140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:18.849534 9140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:18.849573 9140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:18.850606 9140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:18.850641 9140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:18.850674 9140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:14:18.851938 9140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:18.852185 9140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:18.852358 9140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:18.857051 9156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:18.857048 9153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:18.857095 9151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:18.857281 9140 server_base.cc:1034] running on GCE node
I20260809 23:14:18.857487 9140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:18.857736 9140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:18.858923 9140 hybrid_clock.cc:648] HybridClock initialized: now 1786317258858930 us; error 68 us; skew 500 ppm
Aug 09 23:14:18 dist-test-slave-7vjv krb5kdc[8435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317258, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:18.861133 9140 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:14:18.861488 9140 webserver.cc:460] Webserver started at http://127.2.76.195:46739/ using document root <none> and password file <none>
I20260809 23:14:18.861683 9140 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:18.861742 9140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:18.861872 9140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:18.862702 9140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "6bde8bb1c411418b80d80af97176f6c9"
format_stamp: "Formatted at 2026-08-09 23:14:18 on dist-test-slave-7vjv"
server_key: "102e8c30a5f764b5f8d9134d6484bc2f"
server_key_iv: "a73e53f911c1aa2350826b7bf5b0b451"
server_key_version: "encryptionkey@0"
I20260809 23:14:18.863016 9140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "6bde8bb1c411418b80d80af97176f6c9"
format_stamp: "Formatted at 2026-08-09 23:14:18 on dist-test-slave-7vjv"
server_key: "102e8c30a5f764b5f8d9134d6484bc2f"
server_key_iv: "a73e53f911c1aa2350826b7bf5b0b451"
server_key_version: "encryptionkey@0"
I20260809 23:14:18.864203 9140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:14:18.864809 9166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:18.864971 9140 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:18.865058 9140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "6bde8bb1c411418b80d80af97176f6c9"
format_stamp: "Formatted at 2026-08-09 23:14:18 on dist-test-slave-7vjv"
server_key: "102e8c30a5f764b5f8d9134d6484bc2f"
server_key_iv: "a73e53f911c1aa2350826b7bf5b0b451"
server_key_version: "encryptionkey@0"
I20260809 23:14:18.865132 9140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:18.881624 9140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:18.883046 9140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:18.883180 9140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:18.883383 9140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:18.883638 9140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:18.883693 9140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:18.883723 9140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:18.883746 9140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:18.889375 9140 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:39069
I20260809 23:14:18.889410 9367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:39069 every 8 connection(s)
I20260809 23:14:18.889668 9140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:14:18.890066 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 9140
I20260809 23:14:18.890173 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:14:18.890343 2355 external_mini_cluster.cc:1428] Setting key 3a04a61a8fdd4e9fd2f339674eae9605
Aug 09 23:14:18 dist-test-slave-7vjv krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317258, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:18.897585 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:18.890307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:36689 (local address 127.2.76.254:37987)
0809 23:14:18.890359 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:18.890362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:18.890543 (+ 181us) server_negotiation.cc:407] Connection header received
0809 23:14:18.891487 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:18.891488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:18.891505 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:18.891550 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:18.892059 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.892521 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.893389 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.893577 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.893658 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:18.895482 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:18.895483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:18.895483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:18.895489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:18.896564 (+ 1075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:18.896881 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:18.896881 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:18.896882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:18.896926 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:18.897121 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:18.897121 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:18.897121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:18.897209 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:18.897270 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:18.897464 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 23:14:18.897529 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:14:18.897735 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:18.890282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37987 (local address 127.2.76.195:36689)
0809 23:14:18.890424 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:18.890458 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:18.891392 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:18.891606 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:18.891609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:18.891818 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:18.891966 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:18.891969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.892611 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.892612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:18.893303 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:18.893305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.893668 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.893669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:18.893780 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:18.894204 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:18.894224 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:18.895405 (+ 1181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:18.896634 (+ 1229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:18.896640 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:18.896641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:18.896822 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:18.896978 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:18.896979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:18.896979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:18.897064 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:18.897273 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:18.897277 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:18.897367 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 23:14:18.897436 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 23:14:18.897544 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260809 23:14:18.898192 9370 heartbeater.cc:347] Connected to a master server at 127.2.76.254:37987
I20260809 23:14:18.898258 9370 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:18.898412 9370 heartbeater.cc:511] Master 127.2.76.254:37987 requested a full tablet report, sending...
I20260809 23:14:18.898741 8524 ts_manager.cc:194] Registered new tserver with Master: 6bde8bb1c411418b80d80af97176f6c9 (127.2.76.195:39069)
I20260809 23:14:18.899020 8524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:36689
I20260809 23:14:18.901921 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 23:14:18 dist-test-slave-7vjv krb5kdc[8435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 23:14:18 dist-test-slave-7vjv krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.193@KRBTEST.COM
I20260809 23:14:18.923048 9381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:18.916596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39308 (local address 127.2.76.193:40955)
0809 23:14:18.916745 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:18.916749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:18.916765 (+ 16us) server_negotiation.cc:407] Connection header received
0809 23:14:18.916804 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:18.916805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:18.916817 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:18.916867 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:18.917328 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.917911 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.918576 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.918839 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.918912 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:18.920851 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:18.920857 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:18.920858 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:18.920864 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:18.922095 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:18.922410 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:18.922413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:18.922413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:18.922464 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:18.922630 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:18.922630 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:18.922631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:18.922746 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:18.922837 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:18.922911 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 23:14:18.922969 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
Aug 09 23:14:18 dist-test-slave-7vjv krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:18.929523 8577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:18.924202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40046 (local address 127.2.76.254:37987)
0809 23:14:18.924237 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:18.924239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:18.924303 (+ 64us) server_negotiation.cc:407] Connection header received
0809 23:14:18.924333 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:18.924334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:18.924340 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:18.924375 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:18.924746 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.925114 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.925647 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:18.925788 (+ 141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:18.925828 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:18.927500 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:18.927501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:18.927501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:18.927507 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:18.928738 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:18.929000 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:18.929000 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:18.929001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:18.929043 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:18.929171 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:18.929171 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:18.929171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:18.929269 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:18.929324 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:18.929396 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 23:14:18.929443 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 23:14:18.929726 8524 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40046
I20260809 23:14:18.930073 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 8615
I20260809 23:14:18.945729 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 8879
I20260809 23:14:18.954236 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 9140
I20260809 23:14:18.969250 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 8470
2026-08-09T23:14:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1866 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:14:19 dist-test-slave-7vjv krb5kdc[9389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:14:19 dist-test-slave-7vjv krb5kdc[9389](info): set up 2 sockets
Aug 09 23:14:19 dist-test-slave-7vjv krb5kdc[9389](info): commencing operation
krb5kdc: starting...
W20260809 23:14:20.096235 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:14:20 dist-test-slave-7vjv krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:14:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:14:20.201057 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:43759
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:43759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:20.273864 9411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:20.274034 9411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:20.274055 9411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:20.275250 9411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:20.275322 9411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:20.275334 9411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:20.275345 9411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:20.275357 9411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:14:20.276721 9411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:43759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:43759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:20.276913 9411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:20.277088 9411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:20.281543 9427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:20.281585 9421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:20.281602 9422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:20.281625 9411 server_base.cc:1034] running on GCE node
I20260809 23:14:20.282016 9411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:20.282294 9411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:20.283453 9411 hybrid_clock.cc:648] HybridClock initialized: now 1786317260283431 us; error 34 us; skew 500 ppm
Aug 09 23:14:20 dist-test-slave-7vjv krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317260, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:20.285878 9411 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:14:20.286192 9411 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:20.286260 9411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:20.286419 9411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:20.287389 9411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "17f3c1154a4d418baa073aa48d47d265"
format_stamp: "Formatted at 2026-08-09 23:14:20 on dist-test-slave-7vjv"
server_key: "935d5b6ab551402c8b5d24c647bc6198"
server_key_iv: "2a1613e32718ae7f3807ac08781f2082"
server_key_version: "encryptionkey@0"
I20260809 23:14:20.287729 9411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "17f3c1154a4d418baa073aa48d47d265"
format_stamp: "Formatted at 2026-08-09 23:14:20 on dist-test-slave-7vjv"
server_key: "935d5b6ab551402c8b5d24c647bc6198"
server_key_iv: "2a1613e32718ae7f3807ac08781f2082"
server_key_version: "encryptionkey@0"
I20260809 23:14:20.289055 9411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:14:20.289779 9442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:20.289942 9411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:14:20.290045 9411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "17f3c1154a4d418baa073aa48d47d265"
format_stamp: "Formatted at 2026-08-09 23:14:20 on dist-test-slave-7vjv"
server_key: "935d5b6ab551402c8b5d24c647bc6198"
server_key_iv: "2a1613e32718ae7f3807ac08781f2082"
server_key_version: "encryptionkey@0"
I20260809 23:14:20.290126 9411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:20.309504 9411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:20.311568 9411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:20.311725 9411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:20.315670 9411 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:43759
I20260809 23:14:20.315696 9532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:43759 every 8 connection(s)
I20260809 23:14:20.315943 9411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:20.316890 9534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:20.318852 9534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265: Bootstrap starting.
I20260809 23:14:20.319384 9534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:20.319694 9534 log.cc:824] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:20.320338 9534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265: No bootstrap required, opened a new log
I20260809 23:14:20.321821 9534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f3c1154a4d418baa073aa48d47d265" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43759 } }
I20260809 23:14:20.321938 9534 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:20.321991 9534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f3c1154a4d418baa073aa48d47d265, State: Initialized, Role: FOLLOWER
I20260809 23:14:20.322121 9534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f3c1154a4d418baa073aa48d47d265" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43759 } }
I20260809 23:14:20.322196 9534 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:20.322244 9534 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:20.322288 9534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:20.322978 9534 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f3c1154a4d418baa073aa48d47d265" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43759 } }
I20260809 23:14:20.323077 9534 leader_election.cc:302] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17f3c1154a4d418baa073aa48d47d265; no voters:
I20260809 23:14:20.323235 9534 leader_election.cc:288] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:20.323298 9537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:20.323436 9537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [term 1 LEADER]: Becoming Leader. State: Replica: 17f3c1154a4d418baa073aa48d47d265, State: Running, Role: LEADER
I20260809 23:14:20.323495 9534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:20.323546 9537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f3c1154a4d418baa073aa48d47d265" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43759 } }
I20260809 23:14:20.323901 9538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17f3c1154a4d418baa073aa48d47d265" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f3c1154a4d418baa073aa48d47d265" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43759 } } }
I20260809 23:14:20.323916 9540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17f3c1154a4d418baa073aa48d47d265. Latest consensus state: current_term: 1 leader_uuid: "17f3c1154a4d418baa073aa48d47d265" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f3c1154a4d418baa073aa48d47d265" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43759 } } }
I20260809 23:14:20.324010 9538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:20.324023 9540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:20.324255 9551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:20.324740 9551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:20.324944 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 9411
I20260809 23:14:20.325063 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:20.325222 2355 external_mini_cluster.cc:1428] Setting key b97771409f7b6a06a1770eec6d964bb2
I20260809 23:14:20.327628 9551 catalog_manager.cc:1347] Generated new cluster ID: 7c6a4e92e070440e9660179b82239b4c
I20260809 23:14:20.327677 9551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:14:20 dist-test-slave-7vjv krb5kdc[9389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:20.334738 9572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:20.325752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50680 (local address 127.2.76.254:43759)
0809 23:14:20.326329 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:20.326337 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:20.326388 (+ 51us) server_negotiation.cc:407] Connection header received
0809 23:14:20.326483 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:20.326488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:20.326529 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:20.326651 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:20.327327 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.328339 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.329058 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.329454 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.329560 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:20.331774 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:20.331777 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:20.331778 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:20.331788 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:20.333303 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:20.333669 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:20.333672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:20.333672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:20.333754 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:20.333972 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:20.333973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:20.333973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:20.334173 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:20.334340 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:20.334408 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 23:14:20.334533 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":56,"threads_started":1}
I20260809 23:14:20.351547 9551 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:20.352224 9551 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:20.356144 9551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265: Generated new TSK 0
I20260809 23:14:20.356328 9551 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:14:20.384893 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:43759
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:20.460932 9580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:20.461099 9580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:20.461119 9580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:20.462129 9580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:20.462167 9580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:20.462188 9580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:14:20.463490 9580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:43759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:20.463730 9580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:20.463953 9580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:20.468396 9589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:20.468422 9593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:20.468422 9588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:20.468468 9580 server_base.cc:1034] running on GCE node
I20260809 23:14:20.468730 9580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:20.468947 9580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:20.470100 9580 hybrid_clock.cc:648] HybridClock initialized: now 1786317260470089 us; error 23 us; skew 500 ppm
Aug 09 23:14:20 dist-test-slave-7vjv krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317260, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:20.472611 9580 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:14:20.472990 9580 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:20.473068 9580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:20.473191 9580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:20.474053 9580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "8494a9b17fd248169e922fd2219b74b6"
format_stamp: "Formatted at 2026-08-09 23:14:20 on dist-test-slave-7vjv"
server_key: "c70669f87ae8fae85d1b820f99719c16"
server_key_iv: "b233ae8e7b06c9412856293e893a98ec"
server_key_version: "encryptionkey@0"
I20260809 23:14:20.474315 9580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "8494a9b17fd248169e922fd2219b74b6"
format_stamp: "Formatted at 2026-08-09 23:14:20 on dist-test-slave-7vjv"
server_key: "c70669f87ae8fae85d1b820f99719c16"
server_key_iv: "b233ae8e7b06c9412856293e893a98ec"
server_key_version: "encryptionkey@0"
I20260809 23:14:20.475378 9580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:14:20.475934 9605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:20.476150 9580 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:20.476238 9580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "8494a9b17fd248169e922fd2219b74b6"
format_stamp: "Formatted at 2026-08-09 23:14:20 on dist-test-slave-7vjv"
server_key: "c70669f87ae8fae85d1b820f99719c16"
server_key_iv: "b233ae8e7b06c9412856293e893a98ec"
server_key_version: "encryptionkey@0"
I20260809 23:14:20.476301 9580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:20.505724 9580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:20.507308 9580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:20.507452 9580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:20.507663 9580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:20.507951 9580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:20.507980 9580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:20.508005 9580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:20.508047 9580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:20.513329 9580 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:36595
I20260809 23:14:20.513372 9796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:36595 every 8 connection(s)
I20260809 23:14:20.513626 9580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:14:20 dist-test-slave-7vjv krb5kdc[9389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317260, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:20.519570 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 9580
I20260809 23:14:20.519680 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:14:20.519852 2355 external_mini_cluster.cc:1428] Setting key ed2c43d250c2d0c27731a825b35bb63c
I20260809 23:14:20.522010 9572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:20.514277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:54221 (local address 127.2.76.254:43759)
0809 23:14:20.514334 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:20.514336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:20.514558 (+ 222us) server_negotiation.cc:407] Connection header received
0809 23:14:20.514947 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:20.514948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:20.514957 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:20.515012 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:20.515862 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.516419 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.517302 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.517464 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.517515 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:20.519293 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:20.519295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:20.519295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:20.519301 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:20.520743 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:20.521188 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:20.521189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:20.521189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:20.521233 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:20.521482 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:20.521483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:20.521483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:20.521586 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:20.521629 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:20.521900 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 23:14:20.521955 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 23:14:20.522275 9800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:20.514261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:43759 (local address 127.2.76.193:54221)
0809 23:14:20.514431 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:20.514468 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:20.514874 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:20.515058 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:20.515061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:20.515274 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:20.515782 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:20.515787 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.516499 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.516500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:20.517236 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:20.517238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.517531 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.517532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:20.517616 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:20.518060 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:20.518067 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:20.519216 (+ 1149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:20.520821 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:20.520828 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:20.520830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:20.521121 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:20.521284 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:20.521285 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:20.521285 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:20.521426 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:20.521643 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:20.521649 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:20.521785 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 23:14:20.521882 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 23:14:20.522039 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
I20260809 23:14:20.522769 9798 heartbeater.cc:347] Connected to a master server at 127.2.76.254:43759
I20260809 23:14:20.522871 9798 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:20.523029 9798 heartbeater.cc:511] Master 127.2.76.254:43759 requested a full tablet report, sending...
I20260809 23:14:20.523530 9471 ts_manager.cc:194] Registered new tserver with Master: 8494a9b17fd248169e922fd2219b74b6 (127.2.76.193:36595)
I20260809 23:14:20.524390 9471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:54221
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:14:20.544886 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:43759
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:20.619804 9803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:20.619979 9803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:20.620000 9803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:20.621012 9803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:20.621055 9803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:20.621109 9803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:14:20.622414 9803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:43759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:20.622624 9803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:20.622869 9803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:20.627281 9808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:20.627279 9809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:20.627347 9803 server_base.cc:1034] running on GCE node
W20260809 23:14:20.627305 9816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:20.627650 9803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:20.627883 9803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:20.629019 9803 hybrid_clock.cc:648] HybridClock initialized: now 1786317260629003 us; error 32 us; skew 500 ppm
Aug 09 23:14:20 dist-test-slave-7vjv krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317260, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:20.631295 9803 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:14:20.631580 9803 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:20.631644 9803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:20.631781 9803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:20.632722 9803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "39ec9641b3d849eaaa3db559de26b2a3"
format_stamp: "Formatted at 2026-08-09 23:14:20 on dist-test-slave-7vjv"
server_key: "d4bd92126fd7438ed2bdd3b485efeba5"
server_key_iv: "608f0be83b78ea545247a856389f7f94"
server_key_version: "encryptionkey@0"
I20260809 23:14:20.633041 9803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "39ec9641b3d849eaaa3db559de26b2a3"
format_stamp: "Formatted at 2026-08-09 23:14:20 on dist-test-slave-7vjv"
server_key: "d4bd92126fd7438ed2bdd3b485efeba5"
server_key_iv: "608f0be83b78ea545247a856389f7f94"
server_key_version: "encryptionkey@0"
I20260809 23:14:20.634325 9803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:20.635017 9829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:20.635178 9803 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:20.635270 9803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "39ec9641b3d849eaaa3db559de26b2a3"
format_stamp: "Formatted at 2026-08-09 23:14:20 on dist-test-slave-7vjv"
server_key: "d4bd92126fd7438ed2bdd3b485efeba5"
server_key_iv: "608f0be83b78ea545247a856389f7f94"
server_key_version: "encryptionkey@0"
I20260809 23:14:20.635341 9803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:20.656366 9803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:20.657814 9803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:20.657951 9803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:20.658205 9803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:20.658466 9803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:20.658519 9803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:20.658548 9803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:20.658571 9803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:20.663834 9803 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:37175
I20260809 23:14:20.663902 10027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:37175 every 8 connection(s)
I20260809 23:14:20.664151 9803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:14:20.668812 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 9803
I20260809 23:14:20.668890 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:14:20.669070 2355 external_mini_cluster.cc:1428] Setting key fe97b83845fd69a4f897f99eafc5c18f
Aug 09 23:14:20 dist-test-slave-7vjv krb5kdc[9389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317260, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:20.672287 9572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:20.664765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:46425 (local address 127.2.76.254:43759)
0809 23:14:20.664819 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:20.664821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:20.664999 (+ 178us) server_negotiation.cc:407] Connection header received
0809 23:14:20.665374 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:20.665375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:20.665386 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:20.665435 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:20.666137 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.666559 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.667350 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.667508 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.667552 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:20.669555 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:20.669556 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:20.669557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:20.669563 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:20.670850 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:20.671304 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:20.671305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:20.671305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:20.671383 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:20.671662 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:20.671663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:20.671663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:20.671796 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:20.671865 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:20.672071 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 23:14:20.672176 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 23:14:20.672310 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:20.664754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:43759 (local address 127.2.76.194:46425)
0809 23:14:20.664892 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:20.664925 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:20.665292 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:20.665474 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:20.665477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:20.665679 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:20.666070 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:20.666075 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.666635 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.666635 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:20.667284 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:20.667286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.667571 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.667571 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:20.667653 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:20.668145 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:20.668153 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:20.669480 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:20.670925 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:20.670932 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:20.670934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:20.671225 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:20.671453 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:20.671454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:20.671454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:20.671594 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:20.671860 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:20.671864 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:20.671956 (+ 92us) client_negotiation.cc:769] Sending connection context
0809 23:14:20.672029 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:14:20.672177 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":79,"threads_started":1}
I20260809 23:14:20.672772 10028 heartbeater.cc:347] Connected to a master server at 127.2.76.254:43759
I20260809 23:14:20.672856 10028 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:20.672991 10028 heartbeater.cc:511] Master 127.2.76.254:43759 requested a full tablet report, sending...
I20260809 23:14:20.673308 9471 ts_manager.cc:194] Registered new tserver with Master: 39ec9641b3d849eaaa3db559de26b2a3 (127.2.76.194:37175)
I20260809 23:14:20.673651 9471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:46425
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:14:20.692919 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:43759
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:20.765297 10033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:20.765450 10033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:20.765470 10033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:20.766543 10033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:20.766592 10033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:20.766614 10033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:14:20.767884 10033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:43759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:20.768052 10033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:20.768280 10033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:20.772722 10045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:20.772732 10041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:20.772753 10038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:20.772886 10033 server_base.cc:1034] running on GCE node
I20260809 23:14:20.773078 10033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:20.773270 10033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:20.774436 10033 hybrid_clock.cc:648] HybridClock initialized: now 1786317260774417 us; error 32 us; skew 500 ppm
Aug 09 23:14:20 dist-test-slave-7vjv krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317260, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:20.776750 10033 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:14:20.777069 10033 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:20.777138 10033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:20.777280 10033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:20.778196 10033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "53b25c4f77a44547880e3d34ce0e7514"
format_stamp: "Formatted at 2026-08-09 23:14:20 on dist-test-slave-7vjv"
server_key: "e0acf2344694c15050b14621305298e7"
server_key_iv: "9c6c34b6ce7870699b1c2ab6c5683b9e"
server_key_version: "encryptionkey@0"
I20260809 23:14:20.778528 10033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "53b25c4f77a44547880e3d34ce0e7514"
format_stamp: "Formatted at 2026-08-09 23:14:20 on dist-test-slave-7vjv"
server_key: "e0acf2344694c15050b14621305298e7"
server_key_iv: "9c6c34b6ce7870699b1c2ab6c5683b9e"
server_key_version: "encryptionkey@0"
I20260809 23:14:20.779733 10033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:14:20.780493 10054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:20.780666 10033 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:20.780758 10033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "53b25c4f77a44547880e3d34ce0e7514"
format_stamp: "Formatted at 2026-08-09 23:14:20 on dist-test-slave-7vjv"
server_key: "e0acf2344694c15050b14621305298e7"
server_key_iv: "9c6c34b6ce7870699b1c2ab6c5683b9e"
server_key_version: "encryptionkey@0"
I20260809 23:14:20.780839 10033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:20.794008 10033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:20.795503 10033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:20.795642 10033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:20.795831 10033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:20.796133 10033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:20.796172 10033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:20.796206 10033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:20.796229 10033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:20.801766 10033 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:43771
I20260809 23:14:20.801795 10238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:43771 every 8 connection(s)
I20260809 23:14:20.802034 10033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:14:20.806829 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 10033
I20260809 23:14:20.806937 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:14:20.807157 2355 external_mini_cluster.cc:1428] Setting key ca86d81e6cbeeb7a7a9b6c0b1a78b2cd
Aug 09 23:14:20 dist-test-slave-7vjv krb5kdc[9389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317260, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:20.810006 9572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:20.802639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:40931 (local address 127.2.76.254:43759)
0809 23:14:20.802680 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:20.802682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:20.802891 (+ 209us) server_negotiation.cc:407] Connection header received
0809 23:14:20.803301 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:20.803303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:20.803315 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:20.803365 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:20.804155 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.804583 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.805425 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.805596 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.805671 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:20.807734 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:20.807735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:20.807735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:20.807742 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:20.809114 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:20.809419 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:20.809420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:20.809420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:20.809468 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:20.809600 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:20.809600 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:20.809600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:20.809674 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:20.809714 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:20.809895 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 23:14:20.809951 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:14:20.810130 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:20.802609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:43759 (local address 127.2.76.195:40931)
0809 23:14:20.802779 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:20.802827 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:20.803203 (+ 376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:20.803427 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:20.803430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:20.803655 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:20.804045 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:20.804049 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.804670 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.804671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:20.805342 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:20.805344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.805684 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.805685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:20.805821 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:20.806257 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:20.806271 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:20.807613 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:20.809176 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:20.809181 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:20.809183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:20.809359 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:20.809502 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:20.809503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:20.809503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:20.809570 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:20.809716 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:20.809718 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:20.809803 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 23:14:20.809866 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 23:14:20.809957 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":93,"threads_started":1}
I20260809 23:14:20.810552 10239 heartbeater.cc:347] Connected to a master server at 127.2.76.254:43759
I20260809 23:14:20.810638 10239 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:20.810784 10239 heartbeater.cc:511] Master 127.2.76.254:43759 requested a full tablet report, sending...
I20260809 23:14:20.811117 9471 ts_manager.cc:194] Registered new tserver with Master: 53b25c4f77a44547880e3d34ce0e7514 (127.2.76.195:43771)
I20260809 23:14:20.811465 9471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:40931
I20260809 23:14:20.818694 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:14:20.825174 9572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:20.820042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50696 (local address 127.2.76.254:43759)
0809 23:14:20.820133 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:20.820135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:20.820222 (+ 87us) server_negotiation.cc:407] Connection header received
0809 23:14:20.820245 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:20.820246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:20.820254 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:20.820306 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:20.820775 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.821145 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.821736 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.821879 (+ 143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.821918 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:20.822805 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:20.822806 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:20.822806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:20.822812 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:20.824255 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:20.824576 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:20.824577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:20.824577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:20.824624 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:20.824809 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:20.824810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:20.824810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:20.824919 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:20.824989 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:20.825050 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 23:14:20.825104 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 23:14:20.826663 9471 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:14:20.828130 9471 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:14:20.833853 10254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:20.829994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:37175 (local address 127.0.0.1:58278)
0809 23:14:20.830309 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:20.830318 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:20.830361 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:20.830595 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:20.830596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:20.830598 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:20.830639 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:20.830641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.831847 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.831848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:20.832607 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:20.832610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.833560 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.833561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:20.833640 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:20.833656 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 23:14:20.833720 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 23:14:20.833773 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":27,"thread_start_us":58,"threads_started":1}
I20260809 23:14:20.833999 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:20.830315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37830 (local address 127.2.76.193:36595)
0809 23:14:20.830701 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:20.830705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:20.830717 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:14:20.830751 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:20.830753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:20.830766 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:20.830851 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:20.831067 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.831717 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.833054 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.833627 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.833702 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:20.833757 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:20.833863 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 23:14:20.833905 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":72,"threads_started":1}
I20260809 23:14:20.834196 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:20.830242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58278 (local address 127.2.76.194:37175)
0809 23:14:20.830421 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:20.830425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:20.830433 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:20.830469 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:20.830471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:20.830501 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:20.830575 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:20.830717 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.831753 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.832688 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.833447 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.833540 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:20.833590 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:20.834056 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 23:14:20.834103 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":82,"threads_started":1}
I20260809 23:14:20.834245 10253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:20.829957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:43771 (local address 127.0.0.1:56604)
0809 23:14:20.830057 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:20.830092 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:20.830142 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:20.830468 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:20.830470 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:20.830471 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:20.830532 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:20.830533 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.831741 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.831742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:20.833113 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:20.833115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.833997 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.833998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:20.834077 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:20.834081 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:14:20.834133 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 23:14:20.834181 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":84,"thread_start_us":57,"threads_started":1}
I20260809 23:14:20.834506 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:20.830029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56604 (local address 127.2.76.195:43771)
0809 23:14:20.830142 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:20.830146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:20.830157 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:14:20.830212 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:20.830214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:20.830229 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:20.830304 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:20.830637 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.831630 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.833170 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.833844 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.833903 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:20.833963 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:20.834396 (+ 433us) server_negotiation.cc:299] Negotiation successful
0809 23:14:20.834431 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":64,"threads_started":1}
I20260809 23:14:20.834529 10256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:20.830143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:36595 (local address 127.0.0.1:37830)
0809 23:14:20.830391 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:20.830402 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:20.830447 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:20.830944 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:20.830945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:20.830947 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:20.830982 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:20.830983 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.831810 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.831811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:20.832957 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:20.832959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:20.833715 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:20.833716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:20.833779 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:20.833783 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:14:20.833850 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 23:14:20.834467 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":39,"thread_start_us":45,"threads_started":1}
I20260809 23:14:20.834829 9927 tablet_service.cc:1467] Processing CreateTablet for tablet b78f2fa63c514a4aa2a5125c8c94aea7 (DEFAULT_TABLE table=test-table [id=22e3c84d16a04b1e8b7a2ccee7f1f9e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:20.835206 9927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b78f2fa63c514a4aa2a5125c8c94aea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:20.835247 10134 tablet_service.cc:1467] Processing CreateTablet for tablet b78f2fa63c514a4aa2a5125c8c94aea7 (DEFAULT_TABLE table=test-table [id=22e3c84d16a04b1e8b7a2ccee7f1f9e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:20.835422 9685 tablet_service.cc:1467] Processing CreateTablet for tablet b78f2fa63c514a4aa2a5125c8c94aea7 (DEFAULT_TABLE table=test-table [id=22e3c84d16a04b1e8b7a2ccee7f1f9e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:20.835567 10134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b78f2fa63c514a4aa2a5125c8c94aea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:20.835764 9685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b78f2fa63c514a4aa2a5125c8c94aea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:20.838400 10261 tablet_bootstrap.cc:492] T b78f2fa63c514a4aa2a5125c8c94aea7 P 8494a9b17fd248169e922fd2219b74b6: Bootstrap starting.
I20260809 23:14:20.838536 10262 tablet_bootstrap.cc:492] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514: Bootstrap starting.
I20260809 23:14:20.838565 10264 tablet_bootstrap.cc:492] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3: Bootstrap starting.
I20260809 23:14:20.838979 10261 tablet_bootstrap.cc:654] T b78f2fa63c514a4aa2a5125c8c94aea7 P 8494a9b17fd248169e922fd2219b74b6: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:20.839159 10262 tablet_bootstrap.cc:654] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:20.839159 10264 tablet_bootstrap.cc:654] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:20.839668 10261 log.cc:824] T b78f2fa63c514a4aa2a5125c8c94aea7 P 8494a9b17fd248169e922fd2219b74b6: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:20.839831 10264 log.cc:824] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:20.839831 10262 log.cc:824] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:20.840618 10261 tablet_bootstrap.cc:492] T b78f2fa63c514a4aa2a5125c8c94aea7 P 8494a9b17fd248169e922fd2219b74b6: No bootstrap required, opened a new log
I20260809 23:14:20.840641 10262 tablet_bootstrap.cc:492] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514: No bootstrap required, opened a new log
I20260809 23:14:20.840688 10262 ts_tablet_manager.cc:1397] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:20.840696 10261 ts_tablet_manager.cc:1397] T b78f2fa63c514a4aa2a5125c8c94aea7 P 8494a9b17fd248169e922fd2219b74b6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:20.842087 10262 raft_consensus.cc:348] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ec9641b3d849eaaa3db559de26b2a3" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 37175 } } peers { permanent_uuid: "53b25c4f77a44547880e3d34ce0e7514" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 43771 } } peers { permanent_uuid: "8494a9b17fd248169e922fd2219b74b6" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36595 } }
I20260809 23:14:20.842195 10262 raft_consensus.cc:374] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:20.842212 10262 raft_consensus.cc:729] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53b25c4f77a44547880e3d34ce0e7514, State: Initialized, Role: FOLLOWER
I20260809 23:14:20.842303 10262 consensus_queue.cc:260] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ec9641b3d849eaaa3db559de26b2a3" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 37175 } } peers { permanent_uuid: "53b25c4f77a44547880e3d34ce0e7514" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 43771 } } peers { permanent_uuid: "8494a9b17fd248169e922fd2219b74b6" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36595 } }
I20260809 23:14:20.842259 10261 raft_consensus.cc:348] T b78f2fa63c514a4aa2a5125c8c94aea7 P 8494a9b17fd248169e922fd2219b74b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ec9641b3d849eaaa3db559de26b2a3" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 37175 } } peers { permanent_uuid: "53b25c4f77a44547880e3d34ce0e7514" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 43771 } } peers { permanent_uuid: "8494a9b17fd248169e922fd2219b74b6" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36595 } }
I20260809 23:14:20.842391 10261 raft_consensus.cc:374] T b78f2fa63c514a4aa2a5125c8c94aea7 P 8494a9b17fd248169e922fd2219b74b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:20.842430 10261 raft_consensus.cc:729] T b78f2fa63c514a4aa2a5125c8c94aea7 P 8494a9b17fd248169e922fd2219b74b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8494a9b17fd248169e922fd2219b74b6, State: Initialized, Role: FOLLOWER
I20260809 23:14:20.842554 10262 ts_tablet_manager.cc:1428] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:20.842545 10261 consensus_queue.cc:260] T b78f2fa63c514a4aa2a5125c8c94aea7 P 8494a9b17fd248169e922fd2219b74b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ec9641b3d849eaaa3db559de26b2a3" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 37175 } } peers { permanent_uuid: "53b25c4f77a44547880e3d34ce0e7514" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 43771 } } peers { permanent_uuid: "8494a9b17fd248169e922fd2219b74b6" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36595 } }
I20260809 23:14:20.842572 10239 heartbeater.cc:503] Master 127.2.76.254:43759 was elected leader, sending a full tablet report...
I20260809 23:14:20.842798 10261 ts_tablet_manager.cc:1428] T b78f2fa63c514a4aa2a5125c8c94aea7 P 8494a9b17fd248169e922fd2219b74b6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:20.842816 9798 heartbeater.cc:503] Master 127.2.76.254:43759 was elected leader, sending a full tablet report...
I20260809 23:14:20.843185 10264 tablet_bootstrap.cc:492] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3: No bootstrap required, opened a new log
I20260809 23:14:20.843250 10264 ts_tablet_manager.cc:1397] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 23:14:20.844907 10264 raft_consensus.cc:348] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ec9641b3d849eaaa3db559de26b2a3" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 37175 } } peers { permanent_uuid: "53b25c4f77a44547880e3d34ce0e7514" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 43771 } } peers { permanent_uuid: "8494a9b17fd248169e922fd2219b74b6" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36595 } }
I20260809 23:14:20.845052 10264 raft_consensus.cc:374] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:20.845098 10264 raft_consensus.cc:729] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39ec9641b3d849eaaa3db559de26b2a3, State: Initialized, Role: FOLLOWER
I20260809 23:14:20.845206 10264 consensus_queue.cc:260] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ec9641b3d849eaaa3db559de26b2a3" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 37175 } } peers { permanent_uuid: "53b25c4f77a44547880e3d34ce0e7514" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 43771 } } peers { permanent_uuid: "8494a9b17fd248169e922fd2219b74b6" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36595 } }
I20260809 23:14:20.845515 10028 heartbeater.cc:503] Master 127.2.76.254:43759 was elected leader, sending a full tablet report...
I20260809 23:14:20.845491 10264 ts_tablet_manager.cc:1428] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260809 23:14:20.914626 10029 tablet.cc:2369] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:14:21.014356 9799 tablet.cc:2369] T b78f2fa63c514a4aa2a5125c8c94aea7 P 8494a9b17fd248169e922fd2219b74b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:14:21.025079 10276 raft_consensus.cc:482] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:14:21.025199 10276 raft_consensus.cc:504] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ec9641b3d849eaaa3db559de26b2a3" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 37175 } } peers { permanent_uuid: "53b25c4f77a44547880e3d34ce0e7514" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 43771 } } peers { permanent_uuid: "8494a9b17fd248169e922fd2219b74b6" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36595 } }
I20260809 23:14:21.025568 10276 leader_election.cc:288] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53b25c4f77a44547880e3d34ce0e7514 (127.2.76.195:43771), 8494a9b17fd248169e922fd2219b74b6 (127.2.76.193:36595)
I20260809 23:14:21.028085 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.025689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:43771 (local address 127.2.76.194:44851)
0809 23:14:21.025737 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:21.025747 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:21.025782 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:21.025966 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:21.025967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:21.025969 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:21.026048 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:21.026049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.026642 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.026644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:21.027308 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:21.027310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.027831 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.027832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:21.027912 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:21.027915 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:14:21.027978 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 23:14:21.028022 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 23:14:21.028085 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.025776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:44851 (local address 127.2.76.195:43771)
0809 23:14:21.025820 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.025822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.025831 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:14:21.025857 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.025858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.025867 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.025922 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:21.026120 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.026543 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.027399 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.027735 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.027832 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:21.027840 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:21.027984 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 23:14:21.028022 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:14:21.028291 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.025776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:59627 (local address 127.2.76.193:36595)
0809 23:14:21.025812 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.025814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.026112 (+ 298us) server_negotiation.cc:407] Connection header received
0809 23:14:21.026134 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.026135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.026145 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.026202 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:21.026338 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.026757 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.027566 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.027905 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.027979 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:21.027987 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:21.028179 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 23:14:21.028223 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:14:21.028470 10164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b78f2fa63c514a4aa2a5125c8c94aea7" candidate_uuid: "39ec9641b3d849eaaa3db559de26b2a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53b25c4f77a44547880e3d34ce0e7514" is_pre_election: true
I20260809 23:14:21.028517 10277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.025723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:36595 (local address 127.2.76.194:59627)
0809 23:14:21.025875 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:21.025881 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:21.025956 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:21.026207 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:21.026208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:21.026209 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:21.026266 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:21.026267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.026840 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.026841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:21.027478 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:21.027480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.027982 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.027983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:21.028068 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:21.028071 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:14:21.028373 (+ 302us) client_negotiation.cc:240] Negotiation successful
0809 23:14:21.028426 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260809 23:14:21.028609 10164 raft_consensus.cc:2454] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39ec9641b3d849eaaa3db559de26b2a3 in term 0.
I20260809 23:14:21.028856 9833 leader_election.cc:302] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39ec9641b3d849eaaa3db559de26b2a3, 53b25c4f77a44547880e3d34ce0e7514; no voters:
I20260809 23:14:21.028841 9725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b78f2fa63c514a4aa2a5125c8c94aea7" candidate_uuid: "39ec9641b3d849eaaa3db559de26b2a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8494a9b17fd248169e922fd2219b74b6" is_pre_election: true
I20260809 23:14:21.028949 9725 raft_consensus.cc:2454] T b78f2fa63c514a4aa2a5125c8c94aea7 P 8494a9b17fd248169e922fd2219b74b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39ec9641b3d849eaaa3db559de26b2a3 in term 0.
I20260809 23:14:21.028997 10276 raft_consensus.cc:2781] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:14:21.029043 10276 raft_consensus.cc:482] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:14:21.029088 10276 raft_consensus.cc:3037] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:21.029695 10276 raft_consensus.cc:504] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ec9641b3d849eaaa3db559de26b2a3" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 37175 } } peers { permanent_uuid: "53b25c4f77a44547880e3d34ce0e7514" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 43771 } } peers { permanent_uuid: "8494a9b17fd248169e922fd2219b74b6" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36595 } }
I20260809 23:14:21.029825 10276 leader_election.cc:288] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [CANDIDATE]: Term 1 election: Requested vote from peers 53b25c4f77a44547880e3d34ce0e7514 (127.2.76.195:43771), 8494a9b17fd248169e922fd2219b74b6 (127.2.76.193:36595)
I20260809 23:14:21.029945 9725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b78f2fa63c514a4aa2a5125c8c94aea7" candidate_uuid: "39ec9641b3d849eaaa3db559de26b2a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8494a9b17fd248169e922fd2219b74b6"
I20260809 23:14:21.029965 10164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b78f2fa63c514a4aa2a5125c8c94aea7" candidate_uuid: "39ec9641b3d849eaaa3db559de26b2a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53b25c4f77a44547880e3d34ce0e7514"
I20260809 23:14:21.030014 9725 raft_consensus.cc:3037] T b78f2fa63c514a4aa2a5125c8c94aea7 P 8494a9b17fd248169e922fd2219b74b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:21.030030 10164 raft_consensus.cc:3037] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:21.030819 10164 raft_consensus.cc:2454] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39ec9641b3d849eaaa3db559de26b2a3 in term 1.
I20260809 23:14:21.030819 9725 raft_consensus.cc:2454] T b78f2fa63c514a4aa2a5125c8c94aea7 P 8494a9b17fd248169e922fd2219b74b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39ec9641b3d849eaaa3db559de26b2a3 in term 1.
I20260809 23:14:21.030995 9832 leader_election.cc:302] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39ec9641b3d849eaaa3db559de26b2a3, 8494a9b17fd248169e922fd2219b74b6; no voters:
I20260809 23:14:21.031093 10276 raft_consensus.cc:2781] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:21.031219 10276 raft_consensus.cc:686] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [term 1 LEADER]: Becoming Leader. State: Replica: 39ec9641b3d849eaaa3db559de26b2a3, State: Running, Role: LEADER
I20260809 23:14:21.031333 10276 consensus_queue.cc:237] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ec9641b3d849eaaa3db559de26b2a3" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 37175 } } peers { permanent_uuid: "53b25c4f77a44547880e3d34ce0e7514" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 43771 } } peers { permanent_uuid: "8494a9b17fd248169e922fd2219b74b6" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36595 } }
I20260809 23:14:21.032120 9471 catalog_manager.cc:5495] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39ec9641b3d849eaaa3db559de26b2a3 (127.2.76.194). New cstate: current_term: 1 leader_uuid: "39ec9641b3d849eaaa3db559de26b2a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39ec9641b3d849eaaa3db559de26b2a3" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 37175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53b25c4f77a44547880e3d34ce0e7514" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8494a9b17fd248169e922fd2219b74b6" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36595 } health_report { overall_health: UNKNOWN } } }
W20260809 23:14:21.052604 10240 tablet.cc:2369] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:14:21.058221 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.056275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58288 (local address 127.2.76.194:37175)
0809 23:14:21.056331 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.056332 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.056340 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:21.056396 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.056396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.056411 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.056460 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:14:21.056578 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.056955 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.057586 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.057744 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.057785 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:21.057922 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:14:21.058007 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:14:21.058059 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:21.058092 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 23:14:21.058118 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:14:21.059695 10164 raft_consensus.cc:1261] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514 [term 1 FOLLOWER]: Refusing update from remote peer 39ec9641b3d849eaaa3db559de26b2a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:21.059741 9725 raft_consensus.cc:1261] T b78f2fa63c514a4aa2a5125c8c94aea7 P 8494a9b17fd248169e922fd2219b74b6 [term 1 FOLLOWER]: Refusing update from remote peer 39ec9641b3d849eaaa3db559de26b2a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:21.059973 10276 consensus_queue.cc:1035] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8494a9b17fd248169e922fd2219b74b6" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 36595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:21.060120 10279 consensus_queue.cc:1035] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53b25c4f77a44547880e3d34ce0e7514" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:21.062242 10283 mvcc.cc:205] Tried to move back new op lower bound from 7316755501298274304 to 7316755501184815104. Current Snapshot: MvccSnapshot[applied={T|T < 7316755501298274304}]
I20260809 23:14:21.063051 10286 mvcc.cc:205] Tried to move back new op lower bound from 7316755501298274304 to 7316755501184815104. Current Snapshot: MvccSnapshot[applied={T|T < 7316755501298274304}]
I20260809 23:14:21.064774 9468 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:21.064860 9468 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:21.065668 9468 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 17f3c1154a4d418baa073aa48d47d265: Sending DeleteTablet for 3 replicas of tablet b78f2fa63c514a4aa2a5125c8c94aea7
I20260809 23:14:21.065955 9927 tablet_service.cc:1514] Processing DeleteTablet for tablet b78f2fa63c514a4aa2a5125c8c94aea7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:21 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:58278
I20260809 23:14:21.066007 10134 tablet_service.cc:1514] Processing DeleteTablet for tablet b78f2fa63c514a4aa2a5125c8c94aea7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:21 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:56604
I20260809 23:14:21.066011 9685 tablet_service.cc:1514] Processing DeleteTablet for tablet b78f2fa63c514a4aa2a5125c8c94aea7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:21 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:37830
I20260809 23:14:21.066249 10299 tablet_replica.cc:330] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3: stopping tablet replica
I20260809 23:14:21.066334 10299 raft_consensus.cc:2229] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:14:21.066416 10299 raft_consensus.cc:2258] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:21.066447 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 9580
I20260809 23:14:21.066555 10300 tablet_replica.cc:330] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514: stopping tablet replica
I20260809 23:14:21.066638 10300 raft_consensus.cc:2229] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:21.066728 10300 raft_consensus.cc:2258] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:21.069751 10299 ts_tablet_manager.cc:1905] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:21.071513 10300 ts_tablet_manager.cc:1905] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:21.072942 10300 ts_tablet_manager.cc:1918] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:14:21.073024 10300 log.cc:1196] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/b78f2fa63c514a4aa2a5125c8c94aea7
I20260809 23:14:21.073273 10300 ts_tablet_manager.cc:1939] T b78f2fa63c514a4aa2a5125c8c94aea7 P 53b25c4f77a44547880e3d34ce0e7514: Deleting consensus metadata
I20260809 23:14:21.073688 9447 catalog_manager.cc:4841] TS 53b25c4f77a44547880e3d34ce0e7514 (127.2.76.195:43771): tablet b78f2fa63c514a4aa2a5125c8c94aea7 (table test-table [id=22e3c84d16a04b1e8b7a2ccee7f1f9e0]) successfully deleted
I20260809 23:14:21.076198 10299 ts_tablet_manager.cc:1918] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:14:21.076267 10299 log.cc:1196] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/b78f2fa63c514a4aa2a5125c8c94aea7
I20260809 23:14:21.076483 10299 ts_tablet_manager.cc:1939] T b78f2fa63c514a4aa2a5125c8c94aea7 P 39ec9641b3d849eaaa3db559de26b2a3: Deleting consensus metadata
I20260809 23:14:21.078898 9446 catalog_manager.cc:4841] TS 39ec9641b3d849eaaa3db559de26b2a3 (127.2.76.194:37175): tablet b78f2fa63c514a4aa2a5125c8c94aea7 (table test-table [id=22e3c84d16a04b1e8b7a2ccee7f1f9e0]) successfully deleted
W20260809 23:14:21.084968 9446 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:36595 (error 108)
I20260809 23:14:21.085191 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 9803
I20260809 23:14:21.086707 10256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.085355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:36595 (local address 127.0.0.1:37846)
0809 23:14:21.086647 (+ 1292us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:21.086681 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:36595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1276}
W20260809 23:14:21.086820 9446 catalog_manager.cc:4584] TS 8494a9b17fd248169e922fd2219b74b6 (127.2.76.193:36595): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b78f2fa63c514a4aa2a5125c8c94aea7: Network error: Client connection negotiation failed: client connection to 127.2.76.193:36595: connect: Connection refused (error 111)
I20260809 23:14:21.093966 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 10033
I20260809 23:14:21.120689 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 9411
2026-08-09T23:14:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2152 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T23:14:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:21Z Disabled control of system clock
I20260809 23:14:21.151706 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45009
--webserver_interface=127.2.76.254
--webserver_port=0
--builtin_ntp_servers=127.2.76.212:46297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:45009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 23:14:21.227909 10307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:21.228070 10307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:21.228088 10307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:21.229209 10307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:21.229249 10307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:21.229260 10307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:21.229272 10307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:21.229318 10307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:14:21.230509 10307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:46297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:45009
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:21.230669 10307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:21.230952 10307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:21.235388 10314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:21.235428 10307 server_base.cc:1034] running on GCE node
W20260809 23:14:21.235491 10312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:21.235396 10316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:21.235826 10307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:21.236164 10307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:21.237324 10307 hybrid_clock.cc:648] HybridClock initialized: now 1786317261237305 us; error 32 us; skew 500 ppm
I20260809 23:14:21.238472 10307 webserver.cc:460] Webserver started at http://127.2.76.254:41845/ using document root <none> and password file <none>
I20260809 23:14:21.238699 10307 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:21.238765 10307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:21.238891 10307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:21.239833 10307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "fb76c870361f4895a67e0ea1aaadf905"
format_stamp: "Formatted at 2026-08-09 23:14:21 on dist-test-slave-7vjv"
server_key: "774a464cb8185ec99d5df3ccae2d918d"
server_key_iv: "0a84be7b7372b03235cabc0ebbb9eaa4"
server_key_version: "encryptionkey@0"
I20260809 23:14:21.240222 10307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "fb76c870361f4895a67e0ea1aaadf905"
format_stamp: "Formatted at 2026-08-09 23:14:21 on dist-test-slave-7vjv"
server_key: "774a464cb8185ec99d5df3ccae2d918d"
server_key_iv: "0a84be7b7372b03235cabc0ebbb9eaa4"
server_key_version: "encryptionkey@0"
I20260809 23:14:21.241446 10307 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:14:21.242096 10329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:21.242269 10307 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:14:21.242360 10307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "fb76c870361f4895a67e0ea1aaadf905"
format_stamp: "Formatted at 2026-08-09 23:14:21 on dist-test-slave-7vjv"
server_key: "774a464cb8185ec99d5df3ccae2d918d"
server_key_iv: "0a84be7b7372b03235cabc0ebbb9eaa4"
server_key_version: "encryptionkey@0"
I20260809 23:14:21.242440 10307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:21.245611 10307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:21.245855 10307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:21.245980 10307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:21.249707 10307 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:45009
I20260809 23:14:21.249735 10428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:45009 every 8 connection(s)
I20260809 23:14:21.250016 10307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:21.250944 10429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:21.253054 10429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905: Bootstrap starting.
I20260809 23:14:21.253580 10429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:21.253855 10429 log.cc:824] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:21.254585 10429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905: No bootstrap required, opened a new log
I20260809 23:14:21.255795 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 10307
I20260809 23:14:21.255862 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:21.256037 2355 external_mini_cluster.cc:1428] Setting key 5d606c66923274e3b777d9e68407bba7
I20260809 23:14:21.256397 10429 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb76c870361f4895a67e0ea1aaadf905" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45009 } }
I20260809 23:14:21.256619 10429 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:21.256697 10429 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb76c870361f4895a67e0ea1aaadf905, State: Initialized, Role: FOLLOWER
I20260809 23:14:21.256868 10429 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb76c870361f4895a67e0ea1aaadf905" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45009 } }
I20260809 23:14:21.256954 10429 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:21.256986 10429 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:21.257025 10429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:21.257746 10429 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb76c870361f4895a67e0ea1aaadf905" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45009 } }
I20260809 23:14:21.257848 10429 leader_election.cc:302] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb76c870361f4895a67e0ea1aaadf905; no voters:
I20260809 23:14:21.258001 10429 leader_election.cc:288] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:21.258028 10440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:21.258203 10440 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [term 1 LEADER]: Becoming Leader. State: Replica: fb76c870361f4895a67e0ea1aaadf905, State: Running, Role: LEADER
I20260809 23:14:21.258240 10429 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:21.258316 10440 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb76c870361f4895a67e0ea1aaadf905" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45009 } }
I20260809 23:14:21.258558 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.256615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56146 (local address 127.2.76.254:45009)
0809 23:14:21.256828 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.256836 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.256865 (+ 29us) server_negotiation.cc:407] Connection header received
0809 23:14:21.256971 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.256979 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.257031 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.257171 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:21.257845 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:21.257848 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:21.257848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:21.257852 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:21.257885 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:21.257899 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:21.257966 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:21.258186 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 23:14:21.258236 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260809 23:14:21.258801 10441 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb76c870361f4895a67e0ea1aaadf905" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb76c870361f4895a67e0ea1aaadf905" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45009 } } }
I20260809 23:14:21.258876 10441 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:21.259019 10441 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb76c870361f4895a67e0ea1aaadf905. Latest consensus state: current_term: 1 leader_uuid: "fb76c870361f4895a67e0ea1aaadf905" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb76c870361f4895a67e0ea1aaadf905" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45009 } } }
I20260809 23:14:21.259068 10441 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:21.259090 10455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:21.259539 10455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:21.260905 10455 catalog_manager.cc:1347] Generated new cluster ID: 62ef35609a734ba6ab3299e062a18d4d
I20260809 23:14:21.260953 10455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:14:21.272614 10455 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:21.272751 10455 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:21.275457 10455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905: Generated new TSK 0
I20260809 23:14:21.275616 10455 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:14:21.282196 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45009
--builtin_ntp_servers=127.2.76.212:46297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 23:14:21.360518 10472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:21.360702 10472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:21.360724 10472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:21.361714 10472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:21.361752 10472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:21.361776 10472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:14:21.362936 10472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:46297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:21.363173 10472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:21.363451 10472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:21.368637 10480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:21.368757 10482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:21.368666 10484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:21.369061 10472 server_base.cc:1034] running on GCE node
I20260809 23:14:21.369232 10472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:21.369460 10472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:21.370646 10472 hybrid_clock.cc:648] HybridClock initialized: now 1786317261370626 us; error 37 us; skew 500 ppm
I20260809 23:14:21.371853 10472 webserver.cc:460] Webserver started at http://127.2.76.193:38477/ using document root <none> and password file <none>
I20260809 23:14:21.372159 10472 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:21.372238 10472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:21.372362 10472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:21.373281 10472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "1991a2f023ba4c62b75f346778e90aca"
format_stamp: "Formatted at 2026-08-09 23:14:21 on dist-test-slave-7vjv"
server_key: "a29ee2d8ad5e837f3827c7b9f2333335"
server_key_iv: "a7878fee5b1b21666c45c3689885df9c"
server_key_version: "encryptionkey@0"
I20260809 23:14:21.373631 10472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "1991a2f023ba4c62b75f346778e90aca"
format_stamp: "Formatted at 2026-08-09 23:14:21 on dist-test-slave-7vjv"
server_key: "a29ee2d8ad5e837f3827c7b9f2333335"
server_key_iv: "a7878fee5b1b21666c45c3689885df9c"
server_key_version: "encryptionkey@0"
I20260809 23:14:21.374982 10472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:14:21.375658 10490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:21.375859 10472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:21.375983 10472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "1991a2f023ba4c62b75f346778e90aca"
format_stamp: "Formatted at 2026-08-09 23:14:21 on dist-test-slave-7vjv"
server_key: "a29ee2d8ad5e837f3827c7b9f2333335"
server_key_iv: "a7878fee5b1b21666c45c3689885df9c"
server_key_version: "encryptionkey@0"
I20260809 23:14:21.376092 10472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:21.379138 10472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:21.379375 10472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:21.379509 10472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:21.379730 10472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:21.380010 10472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:21.380061 10472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:21.380110 10472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:21.380136 10472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:21.385370 10472 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:46185
I20260809 23:14:21.385437 10687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:46185 every 8 connection(s)
I20260809 23:14:21.385700 10472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:14:21.387446 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 10472
I20260809 23:14:21.387559 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:14:21.387770 2355 external_mini_cluster.cc:1428] Setting key 88b4c8f28774a955120ded93d819191f
I20260809 23:14:21.388690 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.386339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:46709 (local address 127.2.76.254:45009)
0809 23:14:21.386377 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.386379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.386579 (+ 200us) server_negotiation.cc:407] Connection header received
0809 23:14:21.386992 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.386994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.387000 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.387073 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:21.388457 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:21.388458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:21.388458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:21.388463 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:21.388479 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:21.388484 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:21.388542 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:21.388633 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 23:14:21.388645 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 23:14:21.388881 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45009
--builtin_ntp_servers=127.2.76.212:46297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 23:14:21.388906 10694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.386295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45009 (local address 127.2.76.193:46709)
0809 23:14:21.386462 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:21.386495 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:21.386870 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:21.387118 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:21.387121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:21.387128 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:21.388259 (+ 1131us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:14:21.388260 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:21.388284 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:14:21.388309 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:21.388554 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:21.388561 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:21.388563 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:14:21.388636 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:14:21.388691 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260809 23:14:21.390744 10691 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45009
I20260809 23:14:21.390825 10691 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:21.390972 10691 heartbeater.cc:511] Master 127.2.76.254:45009 requested a full tablet report, sending...
I20260809 23:14:21.391453 10362 ts_manager.cc:194] Registered new tserver with Master: 1991a2f023ba4c62b75f346778e90aca (127.2.76.193:46185)
W20260809 23:14:21.461730 10696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:21.461912 10696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:21.461932 10696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:21.462852 10696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:21.462888 10696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:21.462911 10696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:14:21.463999 10696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:46297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:21.464205 10696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:21.464435 10696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:21.468598 10708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:21.468598 10702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:21.468613 10703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:21.468940 10696 server_base.cc:1034] running on GCE node
I20260809 23:14:21.469106 10696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:21.469266 10696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:21.470412 10696 hybrid_clock.cc:648] HybridClock initialized: now 1786317261470400 us; error 27 us; skew 500 ppm
I20260809 23:14:21.471598 10696 webserver.cc:460] Webserver started at http://127.2.76.194:43527/ using document root <none> and password file <none>
I20260809 23:14:21.471869 10696 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:21.471921 10696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:21.472142 10696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:21.473137 10696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "1a3901fbd4e946618cd93c4d0702955c"
format_stamp: "Formatted at 2026-08-09 23:14:21 on dist-test-slave-7vjv"
server_key: "51f23f57fb88006f661488fda6df3635"
server_key_iv: "47d287039e778c66d61f807cc27e722a"
server_key_version: "encryptionkey@0"
I20260809 23:14:21.473469 10696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "1a3901fbd4e946618cd93c4d0702955c"
format_stamp: "Formatted at 2026-08-09 23:14:21 on dist-test-slave-7vjv"
server_key: "51f23f57fb88006f661488fda6df3635"
server_key_iv: "47d287039e778c66d61f807cc27e722a"
server_key_version: "encryptionkey@0"
I20260809 23:14:21.474918 10696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:14:21.475600 10723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:21.475756 10696 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:14:21.475847 10696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "1a3901fbd4e946618cd93c4d0702955c"
format_stamp: "Formatted at 2026-08-09 23:14:21 on dist-test-slave-7vjv"
server_key: "51f23f57fb88006f661488fda6df3635"
server_key_iv: "47d287039e778c66d61f807cc27e722a"
server_key_version: "encryptionkey@0"
I20260809 23:14:21.475894 10696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:21.478801 10696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:21.479012 10696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:21.479140 10696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:21.479337 10696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:21.479610 10696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:21.479642 10696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:21.479694 10696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:21.479719 10696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:21.485086 10696 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:33833
I20260809 23:14:21.485155 10927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:33833 every 8 connection(s)
I20260809 23:14:21.485411 10696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:14:21.488010 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.485982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:39321 (local address 127.2.76.254:45009)
0809 23:14:21.486044 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.486046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.486270 (+ 224us) server_negotiation.cc:407] Connection header received
0809 23:14:21.486699 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.486701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.486707 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.486754 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:21.487722 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:21.487724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:21.487724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:21.487728 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:21.487745 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:21.487749 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:21.487802 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:21.487897 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 23:14:21.487923 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:14:21.488233 10930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.485977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45009 (local address 127.2.76.194:39321)
0809 23:14:21.486154 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:21.486179 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:21.486592 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:21.486810 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:21.486813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:21.486850 (+ 37us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:21.487582 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:14:21.487582 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:21.487605 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:14:21.487627 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:21.487804 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:21.487809 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:21.487812 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:14:21.487907 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 23:14:21.487996 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260809 23:14:21.488727 10928 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45009
I20260809 23:14:21.488799 10928 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:21.488984 10928 heartbeater.cc:511] Master 127.2.76.254:45009 requested a full tablet report, sending...
I20260809 23:14:21.489409 10362 ts_manager.cc:194] Registered new tserver with Master: 1a3901fbd4e946618cd93c4d0702955c (127.2.76.194:33833)
I20260809 23:14:21.494227 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 10696
I20260809 23:14:21.494315 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:14:21.494484 2355 external_mini_cluster.cc:1428] Setting key 7bd8157dd1a22a454c3ea2d78cf51c1f
I20260809 23:14:21.495458 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45009
--builtin_ntp_servers=127.2.76.212:46297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 23:14:21.567281 10933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:21.567432 10933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:21.567451 10933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:21.568435 10933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:21.568470 10933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:21.568514 10933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:14:21.569643 10933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:46297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:21.569856 10933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:21.570101 10933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:21.574609 10948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:21.574620 10941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:21.574597 10943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:21.574855 10933 server_base.cc:1034] running on GCE node
I20260809 23:14:21.575037 10933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:21.575268 10933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:21.576417 10933 hybrid_clock.cc:648] HybridClock initialized: now 1786317261576400 us; error 42 us; skew 500 ppm
I20260809 23:14:21.577452 10933 webserver.cc:460] Webserver started at http://127.2.76.195:44407/ using document root <none> and password file <none>
I20260809 23:14:21.577672 10933 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:21.577731 10933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:21.577844 10933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:21.578867 10933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "f128588e41ff473cbc28aae401469a17"
format_stamp: "Formatted at 2026-08-09 23:14:21 on dist-test-slave-7vjv"
server_key: "52319522f7c041eb6a657882b4953b87"
server_key_iv: "f5984d2c4c6b9b6607f025f9d1645188"
server_key_version: "encryptionkey@0"
I20260809 23:14:21.579236 10933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "f128588e41ff473cbc28aae401469a17"
format_stamp: "Formatted at 2026-08-09 23:14:21 on dist-test-slave-7vjv"
server_key: "52319522f7c041eb6a657882b4953b87"
server_key_iv: "f5984d2c4c6b9b6607f025f9d1645188"
server_key_version: "encryptionkey@0"
I20260809 23:14:21.580653 10933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:14:21.581267 10957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:21.581421 10933 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:21.581511 10933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "f128588e41ff473cbc28aae401469a17"
format_stamp: "Formatted at 2026-08-09 23:14:21 on dist-test-slave-7vjv"
server_key: "52319522f7c041eb6a657882b4953b87"
server_key_iv: "f5984d2c4c6b9b6607f025f9d1645188"
server_key_version: "encryptionkey@0"
I20260809 23:14:21.581578 10933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:21.584401 10933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:21.584619 10933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:21.584739 10933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:21.584940 10933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:21.585196 10933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:21.585243 10933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:21.585276 10933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:21.585300 10933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:21.590309 10933 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:46717
I20260809 23:14:21.590340 11144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:46717 every 8 connection(s)
I20260809 23:14:21.590600 10933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:14:21.593091 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.591188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:55409 (local address 127.2.76.254:45009)
0809 23:14:21.591248 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.591262 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.591437 (+ 175us) server_negotiation.cc:407] Connection header received
0809 23:14:21.591851 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.591853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.591860 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.591903 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:21.592866 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:21.592868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:21.592868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:21.592872 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:21.592887 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:21.592891 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:21.592950 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:21.593014 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 23:14:21.593028 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 23:14:21.593245 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.591163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45009 (local address 127.2.76.195:55409)
0809 23:14:21.591341 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:21.591367 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:21.591738 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:21.591969 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:21.591973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:21.591981 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:21.592736 (+ 755us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:14:21.592737 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:21.592759 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:14:21.592781 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:21.592948 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:21.592953 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:21.592956 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:14:21.593021 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 23:14:21.593069 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260809 23:14:21.593614 11149 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45009
I20260809 23:14:21.593678 11149 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:21.593766 11149 heartbeater.cc:511] Master 127.2.76.254:45009 requested a full tablet report, sending...
I20260809 23:14:21.594024 10362 ts_manager.cc:194] Registered new tserver with Master: f128588e41ff473cbc28aae401469a17 (127.2.76.195:46717)
I20260809 23:14:21.600057 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 10933
I20260809 23:14:21.600173 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:14:21.600332 2355 external_mini_cluster.cc:1428] Setting key 781bbf08ddea6bc1404f52a89ebf11ad
I20260809 23:14:21.601174 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:14:21.603246 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.602341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56156 (local address 127.2.76.254:45009)
0809 23:14:21.602416 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.602418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.602516 (+ 98us) server_negotiation.cc:407] Connection header received
0809 23:14:21.602583 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.602584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.602588 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.602635 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:21.603060 (+ 425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:21.603061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:21.603061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:21.603065 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:21.603076 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:21.603080 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:21.603129 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:21.603177 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 23:14:21.603190 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:14:21.604643 10362 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:14:21.606076 10362 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:14:21.609563 11165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.607997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:46185 (local address 127.0.0.1:45394)
0809 23:14:21.608334 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:21.608343 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:21.608381 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:21.608734 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:21.608735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:21.608737 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:21.609127 (+ 390us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:14:21.609127 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:21.609138 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:14:21.609153 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:21.609397 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:21.609403 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:21.609405 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:14:21.609460 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 23:14:21.609507 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":79,"threads_started":1}
I20260809 23:14:21.609591 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.608224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45394 (local address 127.2.76.193:46185)
0809 23:14:21.608603 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.608607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.608616 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:14:21.608642 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.608644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.608651 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.608718 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:21.609235 (+ 517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:21.609243 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:21.609243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:21.609246 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:21.609275 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:21.609318 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:21.609370 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:21.609506 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 23:14:21.609520 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":85,"threads_started":1}
I20260809 23:14:21.609786 11168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.608191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57954 (local address 127.2.76.195:46717)
0809 23:14:21.608334 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.608338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.608462 (+ 124us) server_negotiation.cc:407] Connection header received
0809 23:14:21.608515 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.608517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.608525 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.608610 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:21.609369 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:21.609376 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:21.609376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:21.609380 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:21.609409 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:21.609432 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:21.609517 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:21.609711 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 23:14:21.609724 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260809 23:14:21.609849 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.608038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54706 (local address 127.2.76.194:33833)
0809 23:14:21.608237 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.608240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.608251 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:14:21.608279 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.608281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.608290 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.608399 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:21.609581 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:21.609590 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:21.609590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:21.609593 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:21.609622 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:21.609645 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:21.609698 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:21.609753 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 23:14:21.609766 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":96,"threads_started":1}
I20260809 23:14:21.609652 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.608191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:46717 (local address 127.0.0.1:57954)
0809 23:14:21.608404 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:21.608413 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:21.608459 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:21.608847 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:21.608848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:21.608850 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:21.609268 (+ 418us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:14:21.609268 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:21.609275 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:14:21.609286 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:21.609540 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:21.609541 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:21.609542 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 23:14:21.609580 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 23:14:21.609593 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":68,"threads_started":1}
I20260809 23:14:21.609807 11160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.607886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:33833 (local address 127.0.0.1:54706)
0809 23:14:21.607980 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:21.608014 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:21.608076 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:21.608403 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:21.608408 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:21.608414 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:21.609489 (+ 1075us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:14:21.609490 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:21.609496 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:14:21.609506 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:21.609698 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:21.609699 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:21.609699 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 23:14:21.609748 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 23:14:21.609759 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":55,"threads_started":1}
I20260809 23:14:21.610478 10573 tablet_service.cc:1467] Processing CreateTablet for tablet 0905b115b88142c49c13646c26173dac (DEFAULT_TABLE table=test-table [id=92c4fd1cede2466d9480f9a0e5014f13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:21.610553 11039 tablet_service.cc:1467] Processing CreateTablet for tablet 0905b115b88142c49c13646c26173dac (DEFAULT_TABLE table=test-table [id=92c4fd1cede2466d9480f9a0e5014f13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:21.610791 10573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0905b115b88142c49c13646c26173dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:21.610826 11039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0905b115b88142c49c13646c26173dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:21.612488 10810 tablet_service.cc:1467] Processing CreateTablet for tablet 0905b115b88142c49c13646c26173dac (DEFAULT_TABLE table=test-table [id=92c4fd1cede2466d9480f9a0e5014f13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:21.612772 10810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0905b115b88142c49c13646c26173dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:21.613370 11171 tablet_bootstrap.cc:492] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca: Bootstrap starting.
I20260809 23:14:21.613652 11172 tablet_bootstrap.cc:492] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17: Bootstrap starting.
I20260809 23:14:21.614219 11171 tablet_bootstrap.cc:654] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:21.614373 11172 tablet_bootstrap.cc:654] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:21.614619 11171 log.cc:824] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:21.614702 11172 log.cc:824] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:21.615315 11171 tablet_bootstrap.cc:492] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca: No bootstrap required, opened a new log
I20260809 23:14:21.615392 11171 ts_tablet_manager.cc:1397] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:21.615458 11172 tablet_bootstrap.cc:492] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17: No bootstrap required, opened a new log
I20260809 23:14:21.615518 11172 ts_tablet_manager.cc:1397] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:14:21.615780 11175 tablet_bootstrap.cc:492] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c: Bootstrap starting.
I20260809 23:14:21.616410 11175 tablet_bootstrap.cc:654] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:21.616698 11175 log.cc:824] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:21.617384 11172 raft_consensus.cc:348] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3901fbd4e946618cd93c4d0702955c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33833 } } peers { permanent_uuid: "f128588e41ff473cbc28aae401469a17" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46717 } } peers { permanent_uuid: "1991a2f023ba4c62b75f346778e90aca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46185 } }
I20260809 23:14:21.617479 11172 raft_consensus.cc:374] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:21.617496 11172 raft_consensus.cc:729] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f128588e41ff473cbc28aae401469a17, State: Initialized, Role: FOLLOWER
I20260809 23:14:21.617511 11175 tablet_bootstrap.cc:492] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c: No bootstrap required, opened a new log
I20260809 23:14:21.617565 11175 ts_tablet_manager.cc:1397] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:14:21.617601 11172 consensus_queue.cc:260] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3901fbd4e946618cd93c4d0702955c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33833 } } peers { permanent_uuid: "f128588e41ff473cbc28aae401469a17" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46717 } } peers { permanent_uuid: "1991a2f023ba4c62b75f346778e90aca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46185 } }
I20260809 23:14:21.617820 11172 ts_tablet_manager.cc:1428] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 23:14:21.617729 11171 raft_consensus.cc:348] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3901fbd4e946618cd93c4d0702955c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33833 } } peers { permanent_uuid: "f128588e41ff473cbc28aae401469a17" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46717 } } peers { permanent_uuid: "1991a2f023ba4c62b75f346778e90aca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46185 } }
I20260809 23:14:21.617856 11171 raft_consensus.cc:374] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:21.617885 11171 raft_consensus.cc:729] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1991a2f023ba4c62b75f346778e90aca, State: Initialized, Role: FOLLOWER
I20260809 23:14:21.617981 11171 consensus_queue.cc:260] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3901fbd4e946618cd93c4d0702955c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33833 } } peers { permanent_uuid: "f128588e41ff473cbc28aae401469a17" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46717 } } peers { permanent_uuid: "1991a2f023ba4c62b75f346778e90aca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46185 } }
I20260809 23:14:21.618211 11171 ts_tablet_manager.cc:1428] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 23:14:21.618257 10691 heartbeater.cc:503] Master 127.2.76.254:45009 was elected leader, sending a full tablet report...
I20260809 23:14:21.618309 11149 heartbeater.cc:503] Master 127.2.76.254:45009 was elected leader, sending a full tablet report...
I20260809 23:14:21.619882 11175 raft_consensus.cc:348] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3901fbd4e946618cd93c4d0702955c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33833 } } peers { permanent_uuid: "f128588e41ff473cbc28aae401469a17" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46717 } } peers { permanent_uuid: "1991a2f023ba4c62b75f346778e90aca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46185 } }
I20260809 23:14:21.620012 11175 raft_consensus.cc:374] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:21.620052 11175 raft_consensus.cc:729] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a3901fbd4e946618cd93c4d0702955c, State: Initialized, Role: FOLLOWER
I20260809 23:14:21.620174 11175 consensus_queue.cc:260] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3901fbd4e946618cd93c4d0702955c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33833 } } peers { permanent_uuid: "f128588e41ff473cbc28aae401469a17" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46717 } } peers { permanent_uuid: "1991a2f023ba4c62b75f346778e90aca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46185 } }
I20260809 23:14:21.620393 11175 ts_tablet_manager.cc:1428] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 23:14:21.620458 10928 heartbeater.cc:503] Master 127.2.76.254:45009 was elected leader, sending a full tablet report...
W20260809 23:14:21.636169 10693 tablet.cc:2369] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:14:21.674610 11178 raft_consensus.cc:482] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:14:21.674711 11178 raft_consensus.cc:504] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3901fbd4e946618cd93c4d0702955c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33833 } } peers { permanent_uuid: "f128588e41ff473cbc28aae401469a17" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46717 } } peers { permanent_uuid: "1991a2f023ba4c62b75f346778e90aca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46185 } }
I20260809 23:14:21.674979 11178 leader_election.cc:288] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a3901fbd4e946618cd93c4d0702955c (127.2.76.194:33833), f128588e41ff473cbc28aae401469a17 (127.2.76.195:46717)
I20260809 23:14:21.676086 10694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.675187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:33833 (local address 127.2.76.193:40413)
0809 23:14:21.675239 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:21.675250 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:21.675306 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:21.675426 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:21.675427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:21.675429 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:21.675786 (+ 357us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:14:21.675787 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:21.675792 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:14:21.675818 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:21.675943 (+ 125us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:21.675944 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:21.675945 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 23:14:21.675991 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 23:14:21.676012 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 23:14:21.676137 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.675281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:40413 (local address 127.2.76.194:33833)
0809 23:14:21.675323 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.675325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.675334 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:14:21.675357 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.675358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.675362 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.675413 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:21.675877 (+ 464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:21.675878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:21.675878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:21.675883 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:21.675897 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:21.675900 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:21.675947 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:21.676019 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 23:14:21.676031 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:14:21.676364 11182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.675292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:46717 (local address 127.2.76.193:35019)
0809 23:14:21.675417 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:21.675425 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:21.675466 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:21.675604 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:21.675605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:21.675607 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:21.675995 (+ 388us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:14:21.675995 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:21.676002 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:14:21.676011 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:21.676219 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:21.676220 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:21.676221 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 23:14:21.676287 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 23:14:21.676299 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260809 23:14:21.676374 11168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.675366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:35019 (local address 127.2.76.195:46717)
0809 23:14:21.675403 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.675404 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.675479 (+ 75us) server_negotiation.cc:407] Connection header received
0809 23:14:21.675516 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.675516 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.675521 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.675574 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:21.676082 (+ 508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:21.676084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:21.676084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:21.676088 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:21.676138 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:21.676142 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:21.676194 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:21.676314 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 23:14:21.676325 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:14:21.676386 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0905b115b88142c49c13646c26173dac" candidate_uuid: "1991a2f023ba4c62b75f346778e90aca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a3901fbd4e946618cd93c4d0702955c" is_pre_election: true
I20260809 23:14:21.676506 10844 raft_consensus.cc:2454] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1991a2f023ba4c62b75f346778e90aca in term 0.
I20260809 23:14:21.676755 10492 leader_election.cc:302] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1991a2f023ba4c62b75f346778e90aca, 1a3901fbd4e946618cd93c4d0702955c; no voters:
I20260809 23:14:21.676849 11075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0905b115b88142c49c13646c26173dac" candidate_uuid: "1991a2f023ba4c62b75f346778e90aca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f128588e41ff473cbc28aae401469a17" is_pre_election: true
I20260809 23:14:21.676889 11178 raft_consensus.cc:2781] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:14:21.676932 11178 raft_consensus.cc:482] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:14:21.676962 11178 raft_consensus.cc:3037] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:21.676971 11075 raft_consensus.cc:2454] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1991a2f023ba4c62b75f346778e90aca in term 0.
I20260809 23:14:21.677629 11178 raft_consensus.cc:504] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3901fbd4e946618cd93c4d0702955c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33833 } } peers { permanent_uuid: "f128588e41ff473cbc28aae401469a17" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46717 } } peers { permanent_uuid: "1991a2f023ba4c62b75f346778e90aca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46185 } }
I20260809 23:14:21.677750 11178 leader_election.cc:288] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [CANDIDATE]: Term 1 election: Requested vote from peers 1a3901fbd4e946618cd93c4d0702955c (127.2.76.194:33833), f128588e41ff473cbc28aae401469a17 (127.2.76.195:46717)
I20260809 23:14:21.677914 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0905b115b88142c49c13646c26173dac" candidate_uuid: "1991a2f023ba4c62b75f346778e90aca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a3901fbd4e946618cd93c4d0702955c"
I20260809 23:14:21.677960 11075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0905b115b88142c49c13646c26173dac" candidate_uuid: "1991a2f023ba4c62b75f346778e90aca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f128588e41ff473cbc28aae401469a17"
I20260809 23:14:21.678037 10844 raft_consensus.cc:3037] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:21.678033 11075 raft_consensus.cc:3037] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:21.678758 10844 raft_consensus.cc:2454] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1991a2f023ba4c62b75f346778e90aca in term 1.
I20260809 23:14:21.678756 11075 raft_consensus.cc:2454] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1991a2f023ba4c62b75f346778e90aca in term 1.
I20260809 23:14:21.678936 10492 leader_election.cc:302] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1991a2f023ba4c62b75f346778e90aca, f128588e41ff473cbc28aae401469a17; no voters:
I20260809 23:14:21.679030 11178 raft_consensus.cc:2781] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:21.679155 11178 raft_consensus.cc:686] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [term 1 LEADER]: Becoming Leader. State: Replica: 1991a2f023ba4c62b75f346778e90aca, State: Running, Role: LEADER
I20260809 23:14:21.679224 11178 consensus_queue.cc:237] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3901fbd4e946618cd93c4d0702955c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33833 } } peers { permanent_uuid: "f128588e41ff473cbc28aae401469a17" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46717 } } peers { permanent_uuid: "1991a2f023ba4c62b75f346778e90aca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46185 } }
I20260809 23:14:21.679992 10362 catalog_manager.cc:5495] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca reported cstate change: term changed from 0 to 1, leader changed from <none> to 1991a2f023ba4c62b75f346778e90aca (127.2.76.193). New cstate: current_term: 1 leader_uuid: "1991a2f023ba4c62b75f346778e90aca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a3901fbd4e946618cd93c4d0702955c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f128588e41ff473cbc28aae401469a17" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1991a2f023ba4c62b75f346778e90aca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46185 } health_report { overall_health: HEALTHY } } }
I20260809 23:14:21.684526 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.683724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45410 (local address 127.2.76.193:46185)
0809 23:14:21.683762 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.683764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.683771 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:14:21.683829 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.683830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.683839 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.683878 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:21.684334 (+ 456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:21.684335 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:21.684336 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:21.684340 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:21.684350 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:21.684353 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:21.684396 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:21.684441 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 23:14:21.684452 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:14:21.685818 10844 raft_consensus.cc:1261] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c [term 1 FOLLOWER]: Refusing update from remote peer 1991a2f023ba4c62b75f346778e90aca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:21.685854 11075 raft_consensus.cc:1261] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17 [term 1 FOLLOWER]: Refusing update from remote peer 1991a2f023ba4c62b75f346778e90aca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:21.686124 11178 consensus_queue.cc:1035] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a3901fbd4e946618cd93c4d0702955c" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 33833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:21.686254 11183 consensus_queue.cc:1035] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [LEADER]: Connected to new peer: Peer: permanent_uuid: "f128588e41ff473cbc28aae401469a17" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 46717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:21.688230 11186 mvcc.cc:205] Tried to move back new op lower bound from 7316755503863066624 to 7316755503838687232. Current Snapshot: MvccSnapshot[applied={T|T < 7316755503863066624}]
I20260809 23:14:21.688724 11187 mvcc.cc:205] Tried to move back new op lower bound from 7316755503863066624 to 7316755503838687232. Current Snapshot: MvccSnapshot[applied={T|T < 7316755503863066624 or (T in {7316755503863066624})}]
I20260809 23:14:21.690622 10362 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:21.690681 10362 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:21.691349 10362 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb76c870361f4895a67e0ea1aaadf905: Sending DeleteTablet for 3 replicas of tablet 0905b115b88142c49c13646c26173dac
I20260809 23:14:21.691641 10573 tablet_service.cc:1514] Processing DeleteTablet for tablet 0905b115b88142c49c13646c26173dac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:21 UTC) from {username='slave'} at 127.0.0.1:45394
I20260809 23:14:21.691681 10810 tablet_service.cc:1514] Processing DeleteTablet for tablet 0905b115b88142c49c13646c26173dac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:21 UTC) from {username='slave'} at 127.0.0.1:54706
I20260809 23:14:21.691888 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 10472
I20260809 23:14:21.691928 11203 tablet_replica.cc:330] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca: stopping tablet replica
I20260809 23:14:21.692022 11203 raft_consensus.cc:2229] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:14:21.692157 11203 raft_consensus.cc:2258] T 0905b115b88142c49c13646c26173dac P 1991a2f023ba4c62b75f346778e90aca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:21.692318 11204 tablet_replica.cc:330] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c: stopping tablet replica
I20260809 23:14:21.692483 11204 raft_consensus.cc:2229] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:21.692505 11039 tablet_service.cc:1514] Processing DeleteTablet for tablet 0905b115b88142c49c13646c26173dac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:21 UTC) from {username='slave'} at 127.0.0.1:57954
I20260809 23:14:21.692610 11204 raft_consensus.cc:2258] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:21.692682 11205 tablet_replica.cc:330] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17: stopping tablet replica
I20260809 23:14:21.692781 11205 raft_consensus.cc:2229] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:21.692883 11205 raft_consensus.cc:2258] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:21.698091 11204 ts_tablet_manager.cc:1905] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:21.698163 11205 ts_tablet_manager.cc:1905] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:21.699456 11204 ts_tablet_manager.cc:1918] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:14:21.699545 11204 log.cc:1196] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/0905b115b88142c49c13646c26173dac
I20260809 23:14:21.699803 11204 ts_tablet_manager.cc:1939] T 0905b115b88142c49c13646c26173dac P 1a3901fbd4e946618cd93c4d0702955c: Deleting consensus metadata
I20260809 23:14:21.700196 10334 catalog_manager.cc:4841] TS 1a3901fbd4e946618cd93c4d0702955c (127.2.76.194:33833): tablet 0905b115b88142c49c13646c26173dac (table test-table [id=92c4fd1cede2466d9480f9a0e5014f13]) successfully deleted
I20260809 23:14:21.700310 11205 ts_tablet_manager.cc:1918] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:14:21.700404 11205 log.cc:1196] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/0905b115b88142c49c13646c26173dac
I20260809 23:14:21.700731 11205 ts_tablet_manager.cc:1939] T 0905b115b88142c49c13646c26173dac P f128588e41ff473cbc28aae401469a17: Deleting consensus metadata
I20260809 23:14:21.701109 10334 catalog_manager.cc:4841] TS f128588e41ff473cbc28aae401469a17 (127.2.76.195:46717): tablet 0905b115b88142c49c13646c26173dac (table test-table [id=92c4fd1cede2466d9480f9a0e5014f13]) successfully deleted
W20260809 23:14:21.709607 10336 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:46185 (error 108)
I20260809 23:14:21.710036 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.709929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:46185 (local address 127.0.0.1:45416)
0809 23:14:21.709980 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:21.710013 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:46185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 23:14:21.710153 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 10696
W20260809 23:14:21.710165 10336 catalog_manager.cc:4584] TS 1991a2f023ba4c62b75f346778e90aca (127.2.76.193:46185): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0905b115b88142c49c13646c26173dac: Network error: Client connection negotiation failed: client connection to 127.2.76.193:46185: connect: Connection refused (error 111)
I20260809 23:14:21.718853 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 10933
I20260809 23:14:21.732177 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 10307
2026-08-09T23:14:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (610 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 23:14:21.754766 2355 mini_oidc.cc:152] Starting JWKS server
I20260809 23:14:21.755861 2355 webserver.cc:460] Webserver started at https://127.0.0.1:44301/ using document root <none> and password file <none>
I20260809 23:14:21.755971 2355 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 23:14:21.756232 2355 webserver.cc:460] Webserver started at http://127.0.0.1:43459/ using document root <none> and password file <none>
2026-08-09T23:14:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:21Z Disabled control of system clock
I20260809 23:14:21.763881 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:42139
--webserver_interface=127.2.76.254
--webserver_port=0
--builtin_ntp_servers=127.2.76.212:44059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:42139
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44301/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/testchainca.cert with env {}
W20260809 23:14:21.835070 11216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:21.835232 11216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:21.835251 11216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:21.836314 11216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:21.836346 11216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:21.836359 11216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:21.836370 11216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:21.836381 11216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 23:14:21.836392 11216 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 23:14:21.836403 11216 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44301/jwks.json
I20260809 23:14:21.837819 11216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:44059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:42139
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:42139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44301/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:21.837976 11216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:21.838096 11216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:21.842089 11232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:21.842114 11229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:21.842137 11230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:21.842142 11216 server_base.cc:1034] running on GCE node
I20260809 23:14:21.842553 11216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:21.842935 11216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:21.844128 11216 hybrid_clock.cc:648] HybridClock initialized: now 1786317261844126 us; error 45 us; skew 500 ppm
I20260809 23:14:21.845155 11216 webserver.cc:460] Webserver started at http://127.2.76.254:45233/ using document root <none> and password file <none>
I20260809 23:14:21.845360 11216 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:21.845400 11216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:21.845534 11216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:21.846370 11216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "05c832fa01a24dc191480ebf13d5c841"
format_stamp: "Formatted at 2026-08-09 23:14:21 on dist-test-slave-7vjv"
server_key: "e0e36bd654f71e593cd6aea254780d42"
server_key_iv: "c3b637453e83c870864bd0af0e8ba727"
server_key_version: "encryptionkey@0"
I20260809 23:14:21.846791 11216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "05c832fa01a24dc191480ebf13d5c841"
format_stamp: "Formatted at 2026-08-09 23:14:21 on dist-test-slave-7vjv"
server_key: "e0e36bd654f71e593cd6aea254780d42"
server_key_iv: "c3b637453e83c870864bd0af0e8ba727"
server_key_version: "encryptionkey@0"
I20260809 23:14:21.848237 11216 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:14:21.848949 11242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:21.849115 11216 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:21.849216 11216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "05c832fa01a24dc191480ebf13d5c841"
format_stamp: "Formatted at 2026-08-09 23:14:21 on dist-test-slave-7vjv"
server_key: "e0e36bd654f71e593cd6aea254780d42"
server_key_iv: "c3b637453e83c870864bd0af0e8ba727"
server_key_version: "encryptionkey@0"
I20260809 23:14:21.849291 11216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:21.860018 11216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:21.860285 11216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:21.860419 11216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:21.863926 11216 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:42139
I20260809 23:14:21.863967 11347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:42139 every 8 connection(s)
I20260809 23:14:21.864282 11216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:21.865131 11349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:21.866959 11349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841: Bootstrap starting.
I20260809 23:14:21.867522 11349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:21.867744 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 11216
I20260809 23:14:21.867794 11349 log.cc:824] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:21.867858 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:21.868016 2355 external_mini_cluster.cc:1428] Setting key cac941fc7edd347316fc84887e522768
I20260809 23:14:21.868484 11349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841: No bootstrap required, opened a new log
I20260809 23:14:21.869858 11349 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c832fa01a24dc191480ebf13d5c841" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 42139 } }
I20260809 23:14:21.869988 11349 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:21.870029 11349 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05c832fa01a24dc191480ebf13d5c841, State: Initialized, Role: FOLLOWER
I20260809 23:14:21.870134 11349 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c832fa01a24dc191480ebf13d5c841" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 42139 } }
I20260809 23:14:21.870208 11349 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:21.870247 11349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:21.870308 11349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:21.870777 11349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c832fa01a24dc191480ebf13d5c841" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 42139 } }
I20260809 23:14:21.870888 11349 leader_election.cc:302] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05c832fa01a24dc191480ebf13d5c841; no voters:
I20260809 23:14:21.871086 11349 leader_election.cc:288] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:21.871119 11357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:21.871214 11357 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [term 1 LEADER]: Becoming Leader. State: Replica: 05c832fa01a24dc191480ebf13d5c841, State: Running, Role: LEADER
I20260809 23:14:21.871289 11357 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c832fa01a24dc191480ebf13d5c841" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 42139 } }
I20260809 23:14:21.871325 11349 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:21.871610 11357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05c832fa01a24dc191480ebf13d5c841. Latest consensus state: current_term: 1 leader_uuid: "05c832fa01a24dc191480ebf13d5c841" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c832fa01a24dc191480ebf13d5c841" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 42139 } } }
I20260809 23:14:21.871687 11357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:21.871624 11359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05c832fa01a24dc191480ebf13d5c841" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05c832fa01a24dc191480ebf13d5c841" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 42139 } } }
I20260809 23:14:21.871853 11359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:21.872596 11369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:21.873025 11369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:21.874347 11369 catalog_manager.cc:1347] Generated new cluster ID: 64b302c6e8ff4559acef3358c8807ece
I20260809 23:14:21.874405 11369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:14:21.874571 11355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.868445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33664 (local address 127.2.76.254:42139)
0809 23:14:21.868629 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.868636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.868667 (+ 31us) server_negotiation.cc:407] Connection header received
0809 23:14:21.869845 (+ 1178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.869850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.869896 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.870015 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:21.870184 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.871463 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.872537 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.873049 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.873167 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:21.873949 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:21.873952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:21.873953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:21.874117 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:21.874146 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:21.874156 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:21.874219 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:21.874283 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 23:14:21.874348 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260809 23:14:21.876850 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:14:21.882829 11369 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:21.883415 11369 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:21.888043 11369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05c832fa01a24dc191480ebf13d5c841: Generated new TSK 0
I20260809 23:14:21.888197 11369 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 23:14:21.912680 11401 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 23:14:21.912813 11401 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 23:14:21.913000 11355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:21.899308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33670 (local address 127.2.76.254:42139)
0809 23:14:21.899345 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:21.899346 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:21.899463 (+ 117us) server_negotiation.cc:407] Connection header received
0809 23:14:21.899547 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:21.899547 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:21.899565 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:21.899617 (+ 52us) server_negotiation.cc:226] Negotiated authn=JWT
0809 23:14:21.899770 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.900237 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.901221 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.901378 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.901418 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:21.901619 (+ 201us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 23:14:21.912808 (+ 11189us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 23:14:21.912944 (+ 136us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33670: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 23:14:21.913059 11355 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33670: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 23:14:21.913522 11400 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:14:21.899224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:42139 (local address 127.0.0.1:33670)
0809 23:14:21.899396 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:21.899406 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:21.899469 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:21.899645 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:21.899646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:21.899648 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 23:14:21.899709 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:21.899711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.900307 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.900308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:21.901124 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:21.901127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:21.901439 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:21.901440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:21.901500 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:21.901521 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 23:14:21.912973 (+ 11452us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 23:14:21.913082 (+ 109us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.76.254:42139: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.2.76.254:42139: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 23:14:21.914076 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 11216
2026-08-09T23:14:21Z chronyd exiting
I20260809 23:14:21.956732 2355 test_util.cc:175] -----------------------------------------------
I20260809 23:14:21.956808 2355 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786317238298104-2355-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (203 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 23:14:21.957518 2355 mini_oidc.cc:152] Starting JWKS server
I20260809 23:14:21.958082 2355 webserver.cc:460] Webserver started at https://127.0.0.1:36781/ using document root <none> and password file <none>
I20260809 23:14:21.958141 2355 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 23:14:21.958282 2355 webserver.cc:460] Webserver started at http://127.0.0.1:39023/ using document root <none> and password file <none>
2026-08-09T23:14:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:21Z Disabled control of system clock
I20260809 23:14:21.966214 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:37703
--webserver_interface=127.2.76.254
--webserver_port=0
--builtin_ntp_servers=127.2.76.212:37921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:37703
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36781/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/testchainca.cert with env {}
W20260809 23:14:22.039016 11415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:22.039177 11415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:22.039196 11415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:22.040241 11415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:22.040278 11415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:22.040289 11415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:22.040300 11415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:22.040346 11415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 23:14:22.040361 11415 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 23:14:22.040380 11415 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36781/jwks.json
I20260809 23:14:22.041653 11415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:37921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:37703
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:37703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36781/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:22.041810 11415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:22.041998 11415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:22.046147 11422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:22.046154 11425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:22.046193 11415 server_base.cc:1034] running on GCE node
W20260809 23:14:22.046154 11427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:22.046531 11415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:22.046828 11415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:22.047996 11415 hybrid_clock.cc:648] HybridClock initialized: now 1786317262047981 us; error 32 us; skew 500 ppm
I20260809 23:14:22.049108 11415 webserver.cc:460] Webserver started at http://127.2.76.254:46111/ using document root <none> and password file <none>
I20260809 23:14:22.049312 11415 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:22.049372 11415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:22.049486 11415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:22.050725 11415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "82630a46f13049a6bb8927442ea95d03"
format_stamp: "Formatted at 2026-08-09 23:14:22 on dist-test-slave-7vjv"
server_key: "8062156847a4789553d755fe5bcf87fb"
server_key_iv: "31790b75246b1c801519c70b4a0d20ac"
server_key_version: "encryptionkey@0"
I20260809 23:14:22.051167 11415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "82630a46f13049a6bb8927442ea95d03"
format_stamp: "Formatted at 2026-08-09 23:14:22 on dist-test-slave-7vjv"
server_key: "8062156847a4789553d755fe5bcf87fb"
server_key_iv: "31790b75246b1c801519c70b4a0d20ac"
server_key_version: "encryptionkey@0"
I20260809 23:14:22.052655 11415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:14:22.053416 11449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:22.053578 11415 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:22.053673 11415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "82630a46f13049a6bb8927442ea95d03"
format_stamp: "Formatted at 2026-08-09 23:14:22 on dist-test-slave-7vjv"
server_key: "8062156847a4789553d755fe5bcf87fb"
server_key_iv: "31790b75246b1c801519c70b4a0d20ac"
server_key_version: "encryptionkey@0"
I20260809 23:14:22.053748 11415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:22.068236 11415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:22.068451 11415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:22.068595 11415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:22.072233 11415 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:37703
I20260809 23:14:22.072278 11534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:37703 every 8 connection(s)
I20260809 23:14:22.072566 11415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:22.073401 11535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:22.075301 11535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03: Bootstrap starting.
I20260809 23:14:22.075847 11535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:22.076076 11535 log.cc:824] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:22.076726 11535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03: No bootstrap required, opened a new log
I20260809 23:14:22.078079 11535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82630a46f13049a6bb8927442ea95d03" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37703 } }
I20260809 23:14:22.078191 11535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:22.078209 11535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82630a46f13049a6bb8927442ea95d03, State: Initialized, Role: FOLLOWER
I20260809 23:14:22.078320 11535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82630a46f13049a6bb8927442ea95d03" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37703 } }
I20260809 23:14:22.078389 11535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:22.078409 11535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:22.078434 11535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:22.078840 11535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82630a46f13049a6bb8927442ea95d03" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37703 } }
I20260809 23:14:22.078912 11535 leader_election.cc:302] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82630a46f13049a6bb8927442ea95d03; no voters:
I20260809 23:14:22.079023 11535 leader_election.cc:288] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:22.079064 11538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:22.079224 11538 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [term 1 LEADER]: Becoming Leader. State: Replica: 82630a46f13049a6bb8927442ea95d03, State: Running, Role: LEADER
I20260809 23:14:22.079233 11535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:22.079324 11538 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82630a46f13049a6bb8927442ea95d03" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37703 } }
I20260809 23:14:22.079715 11542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82630a46f13049a6bb8927442ea95d03. Latest consensus state: current_term: 1 leader_uuid: "82630a46f13049a6bb8927442ea95d03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82630a46f13049a6bb8927442ea95d03" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37703 } } }
I20260809 23:14:22.079795 11539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82630a46f13049a6bb8927442ea95d03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82630a46f13049a6bb8927442ea95d03" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37703 } } }
I20260809 23:14:22.079823 11542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:22.079888 11539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:22.080319 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 11415
I20260809 23:14:22.080447 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:22.080648 2355 external_mini_cluster.cc:1428] Setting key aa483f426d8e52bf79fd7fd471e5add1
I20260809 23:14:22.080077 11553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:22.081337 11553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:22.082762 11553 catalog_manager.cc:1347] Generated new cluster ID: 8f7546f72857403b80033b8de5faba9f
I20260809 23:14:22.082808 11553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:14:22.085799 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:22.081036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35204 (local address 127.2.76.254:37703)
0809 23:14:22.081231 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:22.081239 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:22.081266 (+ 27us) server_negotiation.cc:407] Connection header received
0809 23:14:22.081396 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:22.081402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:22.081461 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:22.081598 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:22.081811 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.082920 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.083921 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.084487 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.084600 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:22.085213 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:22.085217 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:22.085217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:22.085384 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:22.085418 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:22.085428 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:22.085482 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:22.085576 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 23:14:22.085637 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260809 23:14:22.086601 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:14:22.088721 11553 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:22.089257 11553 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:22.101883 11553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82630a46f13049a6bb8927442ea95d03: Generated new TSK 0
I20260809 23:14:22.102025 11553 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 23:14:22.120424 11584 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 23:14:22.120513 11584 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 23:14:22.120813 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:22.108508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35206 (local address 127.2.76.254:37703)
0809 23:14:22.108545 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:22.108547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:22.108844 (+ 297us) server_negotiation.cc:407] Connection header received
0809 23:14:22.108938 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:22.108939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:22.108950 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:22.109004 (+ 54us) server_negotiation.cc:226] Negotiated authn=JWT
0809 23:14:22.109142 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.109624 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.110353 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.110577 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.110645 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:22.110781 (+ 136us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 23:14:22.120601 (+ 9820us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36781/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 23:14:22.120739 (+ 138us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35206: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36781/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 23:14:22.120863 11569 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35206: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36781/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 23:14:22.120893 11579 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:14:22.108488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37703 (local address 127.0.0.1:35206)
0809 23:14:22.108768 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:22.108797 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:22.108859 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:22.109011 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:22.109012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:22.109014 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 23:14:22.109059 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:22.109060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.109708 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.109709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:22.110271 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:22.110272 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.110647 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.110648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:22.110699 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:22.110703 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 23:14:22.120687 (+ 9984us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36781/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 23:14:22.120799 (+ 112us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.76.254:37703: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36781/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":124,"threads_started":1}
I20260809 23:14:22.121188 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 11415
2026-08-09T23:14:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (211 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 23:14:22.169386 2355 mini_oidc.cc:152] Starting JWKS server
I20260809 23:14:22.169974 2355 webserver.cc:460] Webserver started at https://127.0.0.1:45681/ using document root <none> and password file <none>
I20260809 23:14:22.170011 2355 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 23:14:22.170166 2355 webserver.cc:460] Webserver started at http://127.0.0.1:37379/ using document root <none> and password file <none>
2026-08-09T23:14:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:22Z Disabled control of system clock
I20260809 23:14:22.178109 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:44677
--webserver_interface=127.2.76.254
--webserver_port=0
--builtin_ntp_servers=127.2.76.212:42513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:44677
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45681/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 23:14:22.252074 11593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:22.252267 11593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:22.252286 11593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:22.253311 11593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:22.253342 11593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:22.253352 11593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:22.253363 11593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:22.253373 11593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 23:14:22.253383 11593 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 23:14:22.253393 11593 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45681/jwks.json
I20260809 23:14:22.254616 11593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:42513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:44677
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:44677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45681/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:22.254812 11593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:22.255018 11593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:22.259104 11601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:22.259119 11609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:22.259253 11593 server_base.cc:1034] running on GCE node
W20260809 23:14:22.259112 11603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:22.259493 11593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:22.259814 11593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:22.260988 11593 hybrid_clock.cc:648] HybridClock initialized: now 1786317262260976 us; error 36 us; skew 500 ppm
I20260809 23:14:22.262069 11593 webserver.cc:460] Webserver started at http://127.2.76.254:43399/ using document root <none> and password file <none>
I20260809 23:14:22.262274 11593 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:22.262333 11593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:22.262454 11593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:22.263407 11593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "c96385e8b38e4fba9e63c505313dc3ea"
format_stamp: "Formatted at 2026-08-09 23:14:22 on dist-test-slave-7vjv"
server_key: "ee5b65273bbb159c7ea9f2d0207deedc"
server_key_iv: "c0f91ef674dbe8426771de51fdd28274"
server_key_version: "encryptionkey@0"
I20260809 23:14:22.263746 11593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "c96385e8b38e4fba9e63c505313dc3ea"
format_stamp: "Formatted at 2026-08-09 23:14:22 on dist-test-slave-7vjv"
server_key: "ee5b65273bbb159c7ea9f2d0207deedc"
server_key_iv: "c0f91ef674dbe8426771de51fdd28274"
server_key_version: "encryptionkey@0"
I20260809 23:14:22.265093 11593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:14:22.265730 11626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:22.265884 11593 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:14:22.265975 11593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "c96385e8b38e4fba9e63c505313dc3ea"
format_stamp: "Formatted at 2026-08-09 23:14:22 on dist-test-slave-7vjv"
server_key: "ee5b65273bbb159c7ea9f2d0207deedc"
server_key_iv: "c0f91ef674dbe8426771de51fdd28274"
server_key_version: "encryptionkey@0"
I20260809 23:14:22.266053 11593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:22.274600 11593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:22.274840 11593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:22.274971 11593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:22.278720 11593 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:44677
I20260809 23:14:22.278749 11709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:44677 every 8 connection(s)
I20260809 23:14:22.279026 11593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:22.279577 11710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:22.281620 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 11593
I20260809 23:14:22.281723 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:22.281885 2355 external_mini_cluster.cc:1428] Setting key c4714f0d11913fb65483d8fa0a57c4f6
I20260809 23:14:22.282008 11710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea: Bootstrap starting.
I20260809 23:14:22.282709 11710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:22.282991 11710 log.cc:824] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:22.283589 11710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea: No bootstrap required, opened a new log
I20260809 23:14:22.285110 11710 raft_consensus.cc:348] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96385e8b38e4fba9e63c505313dc3ea" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44677 } }
I20260809 23:14:22.285244 11710 raft_consensus.cc:374] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:22.285282 11710 raft_consensus.cc:729] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c96385e8b38e4fba9e63c505313dc3ea, State: Initialized, Role: FOLLOWER
I20260809 23:14:22.285399 11710 consensus_queue.cc:260] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96385e8b38e4fba9e63c505313dc3ea" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44677 } }
I20260809 23:14:22.285488 11710 raft_consensus.cc:388] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:22.285531 11710 raft_consensus.cc:482] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:22.285578 11710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:22.286038 11710 raft_consensus.cc:504] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96385e8b38e4fba9e63c505313dc3ea" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44677 } }
I20260809 23:14:22.286146 11710 leader_election.cc:302] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c96385e8b38e4fba9e63c505313dc3ea; no voters:
I20260809 23:14:22.286288 11710 leader_election.cc:288] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:22.286448 11720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:22.286586 11720 raft_consensus.cc:686] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [term 1 LEADER]: Becoming Leader. State: Replica: c96385e8b38e4fba9e63c505313dc3ea, State: Running, Role: LEADER
I20260809 23:14:22.286538 11710 sys_catalog.cc:564] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:22.286672 11720 consensus_queue.cc:237] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96385e8b38e4fba9e63c505313dc3ea" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44677 } }
I20260809 23:14:22.286846 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:22.282327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40176 (local address 127.2.76.254:44677)
0809 23:14:22.282495 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:22.282501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:22.282534 (+ 33us) server_negotiation.cc:407] Connection header received
0809 23:14:22.282933 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:22.282939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:22.282990 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:22.283119 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:22.283317 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.284151 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.284836 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.285177 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.285304 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:22.285928 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:22.285932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:22.285932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:22.286083 (+ 151us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:22.286110 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:22.286119 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:22.286200 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:22.286279 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 23:14:22.286407 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":61,"threads_started":1}
I20260809 23:14:22.287015 11722 sys_catalog.cc:455] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c96385e8b38e4fba9e63c505313dc3ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96385e8b38e4fba9e63c505313dc3ea" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44677 } } }
I20260809 23:14:22.287088 11722 sys_catalog.cc:458] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:22.287222 11724 sys_catalog.cc:455] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader c96385e8b38e4fba9e63c505313dc3ea. Latest consensus state: current_term: 1 leader_uuid: "c96385e8b38e4fba9e63c505313dc3ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96385e8b38e4fba9e63c505313dc3ea" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44677 } } }
I20260809 23:14:22.287266 11724 sys_catalog.cc:458] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:22.287364 11737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:22.287797 11737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:22.287838 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:14:22.289191 11737 catalog_manager.cc:1347] Generated new cluster ID: e130c028b1354e2e86446c30e3c76553
I20260809 23:14:22.289238 11737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:14:22.300442 11737 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:22.300879 11737 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:22.306876 11737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c96385e8b38e4fba9e63c505313dc3ea: Generated new TSK 0
I20260809 23:14:22.307022 11737 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:14:22.317981 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:22.314867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40182 (local address 127.2.76.254:44677)
0809 23:14:22.314902 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:22.314904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:22.314991 (+ 87us) server_negotiation.cc:407] Connection header received
0809 23:14:22.315075 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:22.315076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:22.315084 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:22.315133 (+ 49us) server_negotiation.cc:226] Negotiated authn=JWT
0809 23:14:22.315267 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.315665 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.316658 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.316831 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.316876 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:22.317054 (+ 178us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 23:14:22.317787 (+ 733us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45681/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 23:14:22.317919 (+ 132us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40182: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45681/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 23:14:22.318028 11716 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40182: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45681/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 23:14:22.318192 11764 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:14:22.314777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:44677 (local address 127.0.0.1:40182)
0809 23:14:22.314925 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:22.314935 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:22.315013 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:22.315155 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:22.315156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:22.315158 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 23:14:22.315210 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:22.315211 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.315746 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.315747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:22.316587 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:22.316591 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.316907 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.316908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:22.316962 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:22.316967 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 23:14:22.317909 (+ 942us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45681/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 23:14:22.318007 (+ 98us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.76.254:44677: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45681/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260809 23:14:22.318380 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 11593
E20260809 23:14:22.340323 11765 webserver.cc:533] Webserver: sslize failed:
E20260809 23:14:22.340390 11765 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T23:14:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (196 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T23:14:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:22Z Disabled control of system clock
I20260809 23:14:22.374141 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:37375
--webserver_interface=127.2.76.254
--webserver_port=0
--builtin_ntp_servers=127.2.76.212:37587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:37375
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/testchainca.cert with env {}
W20260809 23:14:22.450178 11771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:22.450388 11771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:22.450424 11771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:22.451534 11771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:22.451593 11771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:22.451613 11771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:22.451632 11771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:22.451655 11771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 23:14:22.451674 11771 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 23:14:22.451697 11771 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 23:14:22.453022 11771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:37587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:37375
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:37375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:22.453261 11771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:22.453446 11771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:22.457543 11777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:22.457677 11778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:22.457628 11782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:22.457916 11771 server_base.cc:1034] running on GCE node
I20260809 23:14:22.458166 11771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:22.458436 11771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:22.459602 11771 hybrid_clock.cc:648] HybridClock initialized: now 1786317262459590 us; error 27 us; skew 500 ppm
I20260809 23:14:22.460855 11771 webserver.cc:460] Webserver started at http://127.2.76.254:34371/ using document root <none> and password file <none>
I20260809 23:14:22.461117 11771 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:22.461186 11771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:22.461329 11771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:22.462311 11771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "f84ca2b912a14916b366479e36ca101b"
format_stamp: "Formatted at 2026-08-09 23:14:22 on dist-test-slave-7vjv"
server_key: "6ab7014766edd47defc951dc4b34bc68"
server_key_iv: "4ab708b426de86305bf16aa5feecb5f8"
server_key_version: "encryptionkey@0"
I20260809 23:14:22.462631 11771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "f84ca2b912a14916b366479e36ca101b"
format_stamp: "Formatted at 2026-08-09 23:14:22 on dist-test-slave-7vjv"
server_key: "6ab7014766edd47defc951dc4b34bc68"
server_key_iv: "4ab708b426de86305bf16aa5feecb5f8"
server_key_version: "encryptionkey@0"
I20260809 23:14:22.463948 11771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:14:22.464650 11791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:22.464802 11771 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:22.464888 11771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "f84ca2b912a14916b366479e36ca101b"
format_stamp: "Formatted at 2026-08-09 23:14:22 on dist-test-slave-7vjv"
server_key: "6ab7014766edd47defc951dc4b34bc68"
server_key_iv: "4ab708b426de86305bf16aa5feecb5f8"
server_key_version: "encryptionkey@0"
I20260809 23:14:22.464959 11771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:22.480046 11771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:22.480298 11771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:22.480429 11771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:22.483680 11771 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:37375
I20260809 23:14:22.483711 11874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:37375 every 8 connection(s)
I20260809 23:14:22.484012 11771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:22.484964 11875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:22.486847 11875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b: Bootstrap starting.
I20260809 23:14:22.487388 11875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:22.487669 11875 log.cc:824] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:22.488135 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 11771
I20260809 23:14:22.488273 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:22.488420 11875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b: No bootstrap required, opened a new log
I20260809 23:14:22.488435 2355 external_mini_cluster.cc:1428] Setting key 409d2b6d4cc7fe57c5e37bf6611e9642
I20260809 23:14:22.491108 11875 raft_consensus.cc:348] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84ca2b912a14916b366479e36ca101b" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37375 } }
I20260809 23:14:22.491254 11875 raft_consensus.cc:374] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:22.491281 11875 raft_consensus.cc:729] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f84ca2b912a14916b366479e36ca101b, State: Initialized, Role: FOLLOWER
I20260809 23:14:22.491380 11875 consensus_queue.cc:260] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84ca2b912a14916b366479e36ca101b" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37375 } }
I20260809 23:14:22.491459 11875 raft_consensus.cc:388] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:22.491494 11875 raft_consensus.cc:482] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:22.491540 11875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:22.492060 11875 raft_consensus.cc:504] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84ca2b912a14916b366479e36ca101b" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37375 } }
I20260809 23:14:22.492226 11875 leader_election.cc:302] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f84ca2b912a14916b366479e36ca101b; no voters:
I20260809 23:14:22.492379 11875 leader_election.cc:288] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:22.492401 11881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:22.492508 11881 raft_consensus.cc:686] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [term 1 LEADER]: Becoming Leader. State: Replica: f84ca2b912a14916b366479e36ca101b, State: Running, Role: LEADER
I20260809 23:14:22.492584 11881 consensus_queue.cc:237] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84ca2b912a14916b366479e36ca101b" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37375 } }
I20260809 23:14:22.492614 11875 sys_catalog.cc:564] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:22.492892 11882 sys_catalog.cc:455] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f84ca2b912a14916b366479e36ca101b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84ca2b912a14916b366479e36ca101b" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37375 } } }
I20260809 23:14:22.492897 11884 sys_catalog.cc:455] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f84ca2b912a14916b366479e36ca101b. Latest consensus state: current_term: 1 leader_uuid: "f84ca2b912a14916b366479e36ca101b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f84ca2b912a14916b366479e36ca101b" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37375 } } }
I20260809 23:14:22.493033 11882 sys_catalog.cc:458] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:22.493052 11884 sys_catalog.cc:458] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:22.493399 11895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:22.493940 11895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:22.494422 11880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:22.488905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40162 (local address 127.2.76.254:37375)
0809 23:14:22.489109 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:22.489117 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:22.489149 (+ 32us) server_negotiation.cc:407] Connection header received
0809 23:14:22.489754 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:22.489762 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:22.489831 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:22.489970 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:22.490114 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.491386 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.492293 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.492833 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.492934 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:22.493754 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:22.493758 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:22.493758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:14:22.493935 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:22.493972 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:14:22.493986 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:22.494054 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:22.494152 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 23:14:22.494230 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260809 23:14:22.495426 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:14:22.497040 11895 catalog_manager.cc:1347] Generated new cluster ID: 9617635ad61c4e15a0677285d7ef768e
I20260809 23:14:22.497092 11895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:14:22.515054 11895 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:22.515616 11895 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:22.520912 11895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f84ca2b912a14916b366479e36ca101b: Generated new TSK 0
I20260809 23:14:22.521070 11895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:14:23.051445 11880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:22.531746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40174 (local address 127.2.76.254:37375)
0809 23:14:22.531817 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:22.531819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:22.531913 (+ 94us) server_negotiation.cc:407] Connection header received
0809 23:14:22.531975 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:22.531975 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:22.531984 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:22.532040 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0809 23:14:22.532181 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.532639 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.533657 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.533853 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.533899 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:22.534102 (+ 203us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 23:14:23.051134 (+517032us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 23:14:23.051356 (+ 222us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40174: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":33}
W20260809 23:14:23.051532 11880 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40174: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 23:14:23.051542 11917 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:14:22.531666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37375 (local address 127.0.0.1:40174)
0809 23:14:22.531845 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:22.531855 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:22.531915 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:22.532049 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:22.532049 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:22.532051 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 23:14:22.532130 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:22.532132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.532726 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.532728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:22.533581 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:22.533584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:22.533924 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:22.533925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:22.533993 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:22.533998 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 23:14:23.051298 (+517300us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 23:14:23.051424 (+ 126us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.76.254:37375: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":117,"threads_started":1}
I20260809 23:14:23.051925 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 11771
2026-08-09T23:14:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (713 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (87 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (81 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:14:23 dist-test-slave-7vjv krb5kdc[11941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:14:23 dist-test-slave-7vjv krb5kdc[11941](info): set up 2 sockets
Aug 09 23:14:23 dist-test-slave-7vjv krb5kdc[11941](info): commencing operation
krb5kdc: starting...
W20260809 23:14:24.366063 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:14:24 dist-test-slave-7vjv krb5kdc[11941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:14:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:14:24.470211 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:40489
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:40489
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:24.543265 11961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:24.543467 11961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:24.543499 11961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:24.544607 11961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:24.544672 11961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:24.544696 11961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:24.544725 11961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:24.544754 11961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:14:24.546066 11961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:40489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:40489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:24.546320 11961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:24.546571 11961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:24.551162 11975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:24.551283 11961 server_base.cc:1034] running on GCE node
W20260809 23:14:24.551121 11978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:24.551139 11974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:24.551640 11961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:24.551947 11961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:24.553088 11961 hybrid_clock.cc:648] HybridClock initialized: now 1786317264553064 us; error 40 us; skew 500 ppm
Aug 09 23:14:24 dist-test-slave-7vjv krb5kdc[11941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317264, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:24.558027 11961 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:14:24.558434 11961 webserver.cc:460] Webserver started at http://127.2.76.254:35651/ using document root <none> and password file <none>
I20260809 23:14:24.558642 11961 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:24.558687 11961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:24.558830 11961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:24.559818 11961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "09d896125e644a709632e1751e3a1206"
format_stamp: "Formatted at 2026-08-09 23:14:24 on dist-test-slave-7vjv"
server_key: "711c54bf346fcce28cbe52bdd685d78c"
server_key_iv: "9a801a84b6f567188a2548b8810404ed"
server_key_version: "encryptionkey@0"
I20260809 23:14:24.560253 11961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "09d896125e644a709632e1751e3a1206"
format_stamp: "Formatted at 2026-08-09 23:14:24 on dist-test-slave-7vjv"
server_key: "711c54bf346fcce28cbe52bdd685d78c"
server_key_iv: "9a801a84b6f567188a2548b8810404ed"
server_key_version: "encryptionkey@0"
I20260809 23:14:24.561547 11961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:14:24.562343 11990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:24.562532 11961 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:14:24.562628 11961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "09d896125e644a709632e1751e3a1206"
format_stamp: "Formatted at 2026-08-09 23:14:24 on dist-test-slave-7vjv"
server_key: "711c54bf346fcce28cbe52bdd685d78c"
server_key_iv: "9a801a84b6f567188a2548b8810404ed"
server_key_version: "encryptionkey@0"
I20260809 23:14:24.562695 11961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:24.571559 11961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:24.573170 11961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:24.573333 11961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:24.577150 11961 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:40489
I20260809 23:14:24.577212 12072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:40489 every 8 connection(s)
I20260809 23:14:24.577548 11961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:24.578393 12074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:24.580618 12074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206: Bootstrap starting.
I20260809 23:14:24.581146 12074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:24.581429 12074 log.cc:824] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:24.582041 12074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206: No bootstrap required, opened a new log
I20260809 23:14:24.583549 12074 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d896125e644a709632e1751e3a1206" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40489 } }
I20260809 23:14:24.583681 12074 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:24.583724 12074 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d896125e644a709632e1751e3a1206, State: Initialized, Role: FOLLOWER
I20260809 23:14:24.583882 12074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d896125e644a709632e1751e3a1206" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40489 } }
I20260809 23:14:24.583941 12074 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:24.583962 12074 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:24.584022 12074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:24.584363 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 11961
I20260809 23:14:24.584448 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:24.584630 12074 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d896125e644a709632e1751e3a1206" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40489 } }
I20260809 23:14:24.584652 2355 external_mini_cluster.cc:1428] Setting key 5b367e951e45e6c8a6947897fcaffda6
I20260809 23:14:24.584740 12074 leader_election.cc:302] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09d896125e644a709632e1751e3a1206; no voters:
I20260809 23:14:24.584910 12074 leader_election.cc:288] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:24.585259 12074 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:24.584940 12080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:24.585666 12080 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [term 1 LEADER]: Becoming Leader. State: Replica: 09d896125e644a709632e1751e3a1206, State: Running, Role: LEADER
I20260809 23:14:24.585841 12080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d896125e644a709632e1751e3a1206" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40489 } }
I20260809 23:14:24.586258 12081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09d896125e644a709632e1751e3a1206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d896125e644a709632e1751e3a1206" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40489 } } }
I20260809 23:14:24.586345 12081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:24.586488 12091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09d896125e644a709632e1751e3a1206. Latest consensus state: current_term: 1 leader_uuid: "09d896125e644a709632e1751e3a1206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09d896125e644a709632e1751e3a1206" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40489 } } }
I20260809 23:14:24.586655 12111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:24.586824 12091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:24.587244 12111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:24.588673 12111 catalog_manager.cc:1347] Generated new cluster ID: 2522ba3827f84d128803193aedae69b4
I20260809 23:14:24.588726 12111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:14:24 dist-test-slave-7vjv krb5kdc[11941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:24.594810 12085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:24.585379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34136 (local address 127.2.76.254:40489)
0809 23:14:24.585570 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:24.585579 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:24.585609 (+ 30us) server_negotiation.cc:407] Connection header received
0809 23:14:24.585725 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:24.585731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:24.585781 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:24.586789 (+ 1008us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:24.586893 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:24.588041 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:24.588987 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:24.589381 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:24.589500 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:24.591662 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:24.591666 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:24.591666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:24.591677 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:24.593293 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:24.593661 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:24.593664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:24.593664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:24.593754 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:24.593982 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:24.593983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:24.593983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:24.594216 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:24.594394 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:24.594480 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 23:14:24.594610 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":98,"threads_started":1}
I20260809 23:14:24.612421 12111 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:24.613072 12111 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:24.619248 12111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09d896125e644a709632e1751e3a1206: Generated new TSK 0
I20260809 23:14:24.619433 12111 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:14:24.644869 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40489
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:24.737304 12122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:24.737473 12122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:24.737493 12122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:24.738459 12122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:24.738492 12122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:24.738514 12122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:14:24.739890 12122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:24.740057 12122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:24.740272 12122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:24.744673 12132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:24.744690 12129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:24.744886 12128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:24.745007 12122 server_base.cc:1034] running on GCE node
I20260809 23:14:24.745234 12122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:24.745458 12122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:24.746630 12122 hybrid_clock.cc:648] HybridClock initialized: now 1786317264746612 us; error 31 us; skew 500 ppm
Aug 09 23:14:24 dist-test-slave-7vjv krb5kdc[11941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317264, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:24.749063 12122 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:14:24.749465 12122 webserver.cc:460] Webserver started at http://127.2.76.193:38271/ using document root <none> and password file <none>
I20260809 23:14:24.749661 12122 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:24.749719 12122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:24.749837 12122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:24.750738 12122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "697c4eb3c2234fba8a634673ebf8ff60"
format_stamp: "Formatted at 2026-08-09 23:14:24 on dist-test-slave-7vjv"
server_key: "2658443fabef21b6b8274f0cba1089ed"
server_key_iv: "dbc1beab4b1760b1b4e1dc1dba6108da"
server_key_version: "encryptionkey@0"
I20260809 23:14:24.751060 12122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "697c4eb3c2234fba8a634673ebf8ff60"
format_stamp: "Formatted at 2026-08-09 23:14:24 on dist-test-slave-7vjv"
server_key: "2658443fabef21b6b8274f0cba1089ed"
server_key_iv: "dbc1beab4b1760b1b4e1dc1dba6108da"
server_key_version: "encryptionkey@0"
I20260809 23:14:24.752336 12122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:14:24.752967 12146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:24.753130 12122 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:24.753221 12122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "697c4eb3c2234fba8a634673ebf8ff60"
format_stamp: "Formatted at 2026-08-09 23:14:24 on dist-test-slave-7vjv"
server_key: "2658443fabef21b6b8274f0cba1089ed"
server_key_iv: "dbc1beab4b1760b1b4e1dc1dba6108da"
server_key_version: "encryptionkey@0"
I20260809 23:14:24.753283 12122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:24.761839 12122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:24.763929 12122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:24.764072 12122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:24.764336 12122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:24.764609 12122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:24.764659 12122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:24.764694 12122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:24.764721 12122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:24.770252 12122 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:42479
I20260809 23:14:24.770289 12351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:42479 every 8 connection(s)
I20260809 23:14:24.770555 12122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:14:24 dist-test-slave-7vjv krb5kdc[11941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317264, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:24.778159 12085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:24.771171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:49929 (local address 127.2.76.254:40489)
0809 23:14:24.771243 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:24.771245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:24.771398 (+ 153us) server_negotiation.cc:407] Connection header received
0809 23:14:24.772199 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:24.772200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:24.772209 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:24.772250 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:24.772721 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:24.773155 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:24.773985 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:24.774155 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:24.774201 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:24.775966 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:24.775968 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:24.775968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:24.775974 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:24.777050 (+ 1076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:24.777378 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:24.777379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:24.777379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:24.777426 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:24.777676 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:24.777676 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:24.777676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:24.777770 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:24.777820 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:24.778034 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 23:14:24.778098 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 23:14:24.778323 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:24.771136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40489 (local address 127.2.76.193:49929)
0809 23:14:24.771289 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:24.771326 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:24.772061 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:24.772300 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:24.772303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:24.772509 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:24.772657 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:24.772660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:24.773229 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:24.773230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:24.773916 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:24.773918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:24.774223 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:24.774224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:24.774308 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:24.774756 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:24.774763 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:24.775891 (+ 1128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:24.777114 (+ 1223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:24.777120 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:24.777121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:24.777316 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:24.777481 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:24.777481 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:24.777481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:24.777597 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:24.777825 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:24.777830 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:24.777928 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 23:14:24.778007 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 23:14:24.778114 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260809 23:14:24.778923 12352 heartbeater.cc:347] Connected to a master server at 127.2.76.254:40489
I20260809 23:14:24.779014 12352 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:24.779183 12352 heartbeater.cc:511] Master 127.2.76.254:40489 requested a full tablet report, sending...
I20260809 23:14:24.779675 12018 ts_manager.cc:194] Registered new tserver with Master: 697c4eb3c2234fba8a634673ebf8ff60 (127.2.76.193:42479)
I20260809 23:14:24.779791 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 12122
I20260809 23:14:24.779896 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:14:24.780051 2355 external_mini_cluster.cc:1428] Setting key 0c726e1581c50b9c920d6526903aa3c7
I20260809 23:14:24.780385 12018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:49929
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:14:24.804134 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40489
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:24.874583 12361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:24.874768 12361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:24.874806 12361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:24.875823 12361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:24.875880 12361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:24.875921 12361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:14:24.877285 12361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:24.877521 12361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:24.877724 12361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:24.882229 12370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:24.882236 12373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:24.882244 12369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:24.882436 12361 server_base.cc:1034] running on GCE node
I20260809 23:14:24.882618 12361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:24.882831 12361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:24.883999 12361 hybrid_clock.cc:648] HybridClock initialized: now 1786317264883995 us; error 32 us; skew 500 ppm
Aug 09 23:14:24 dist-test-slave-7vjv krb5kdc[11941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317264, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:24.886240 12361 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:14:24.886730 12361 webserver.cc:460] Webserver started at http://127.2.76.194:34277/ using document root <none> and password file <none>
I20260809 23:14:24.886906 12361 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:24.886941 12361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:24.887017 12361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:24.887890 12361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "2c58f03fd3db4de8a3a2305f429bbc30"
format_stamp: "Formatted at 2026-08-09 23:14:24 on dist-test-slave-7vjv"
server_key: "7ac80e9616169a504d4be5587facbe36"
server_key_iv: "72cc3ecec6881f7dac9e884d80bf2e61"
server_key_version: "encryptionkey@0"
I20260809 23:14:24.888250 12361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "2c58f03fd3db4de8a3a2305f429bbc30"
format_stamp: "Formatted at 2026-08-09 23:14:24 on dist-test-slave-7vjv"
server_key: "7ac80e9616169a504d4be5587facbe36"
server_key_iv: "72cc3ecec6881f7dac9e884d80bf2e61"
server_key_version: "encryptionkey@0"
I20260809 23:14:24.889478 12361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:24.890053 12382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:24.890210 12361 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:24.890296 12361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "2c58f03fd3db4de8a3a2305f429bbc30"
format_stamp: "Formatted at 2026-08-09 23:14:24 on dist-test-slave-7vjv"
server_key: "7ac80e9616169a504d4be5587facbe36"
server_key_iv: "72cc3ecec6881f7dac9e884d80bf2e61"
server_key_version: "encryptionkey@0"
I20260809 23:14:24.890370 12361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:24.899092 12361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:24.900691 12361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:24.900830 12361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:24.901103 12361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:24.901381 12361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:24.901434 12361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:24.901474 12361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:24.901501 12361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:24.906909 12361 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:39637
I20260809 23:14:24.906947 12579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:39637 every 8 connection(s)
I20260809 23:14:24.907215 12361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:14:24.907909 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 12361
I20260809 23:14:24.908012 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:14:24.908210 2355 external_mini_cluster.cc:1428] Setting key 50e224bc3c3cb07a6761cf725586941c
Aug 09 23:14:24 dist-test-slave-7vjv krb5kdc[11941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317264, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:24.916718 12085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:24.907917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:55857 (local address 127.2.76.254:40489)
0809 23:14:24.907967 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:24.907968 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:24.908183 (+ 215us) server_negotiation.cc:407] Connection header received
0809 23:14:24.909174 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:24.909174 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:24.909183 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:24.909217 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:24.909943 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:24.910418 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:24.911463 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:24.911722 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:24.911773 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:24.914096 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:24.914097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:24.914097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:24.914104 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:24.915494 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:24.915856 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:24.915857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:24.915857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:24.915910 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:24.916159 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:24.916160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:24.916161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:24.916259 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:24.916329 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:24.916562 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 23:14:24.916639 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 23:14:24.916851 12586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:24.907853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40489 (local address 127.2.76.194:55857)
0809 23:14:24.908056 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:24.908116 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:24.909072 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:24.909286 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:24.909289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:24.909640 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:24.909867 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:24.909873 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:24.910545 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:24.910547 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:24.911371 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:24.911373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:24.911805 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:24.911806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:24.911937 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:24.912446 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:24.912455 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:24.913994 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:24.915589 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:24.915595 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:24.915597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:24.915786 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:24.915971 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:24.915971 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:24.915972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:24.916072 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:24.916322 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:24.916329 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:24.916445 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 23:14:24.916537 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 23:14:24.916665 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
I20260809 23:14:24.917243 12583 heartbeater.cc:347] Connected to a master server at 127.2.76.254:40489
I20260809 23:14:24.917330 12583 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:24.917464 12583 heartbeater.cc:511] Master 127.2.76.254:40489 requested a full tablet report, sending...
I20260809 23:14:24.917732 12016 ts_manager.cc:194] Registered new tserver with Master: 2c58f03fd3db4de8a3a2305f429bbc30 (127.2.76.194:39637)
I20260809 23:14:24.918179 12016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:55857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:14:24.932322 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40489
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:25.005008 12592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:25.005196 12592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:25.005218 12592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:25.006362 12592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:25.006403 12592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:25.006430 12592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:14:25.007685 12592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:25.007856 12592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:25.008013 12592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:25.012717 12610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:25.012851 12604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:25.012717 12600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:25.013096 12592 server_base.cc:1034] running on GCE node
I20260809 23:14:25.013265 12592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:25.013470 12592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:25.014600 12592 hybrid_clock.cc:648] HybridClock initialized: now 1786317265014583 us; error 31 us; skew 500 ppm
Aug 09 23:14:25 dist-test-slave-7vjv krb5kdc[11941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317265, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:25.017086 12592 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:14:25.017437 12592 webserver.cc:460] Webserver started at http://127.2.76.195:44257/ using document root <none> and password file <none>
I20260809 23:14:25.017625 12592 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:25.017664 12592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:25.017787 12592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:25.018638 12592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "87d1ddb3450543768b3812b714cf7fa1"
format_stamp: "Formatted at 2026-08-09 23:14:25 on dist-test-slave-7vjv"
server_key: "adb71cd7823135e182d4ebc6b85347d3"
server_key_iv: "d40f4f63319236ad3ebd32d45fbc2df3"
server_key_version: "encryptionkey@0"
I20260809 23:14:25.018944 12592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "87d1ddb3450543768b3812b714cf7fa1"
format_stamp: "Formatted at 2026-08-09 23:14:25 on dist-test-slave-7vjv"
server_key: "adb71cd7823135e182d4ebc6b85347d3"
server_key_iv: "d40f4f63319236ad3ebd32d45fbc2df3"
server_key_version: "encryptionkey@0"
I20260809 23:14:25.020083 12592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:14:25.020766 12623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:25.020934 12592 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:14:25.021024 12592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "87d1ddb3450543768b3812b714cf7fa1"
format_stamp: "Formatted at 2026-08-09 23:14:25 on dist-test-slave-7vjv"
server_key: "adb71cd7823135e182d4ebc6b85347d3"
server_key_iv: "d40f4f63319236ad3ebd32d45fbc2df3"
server_key_version: "encryptionkey@0"
I20260809 23:14:25.021104 12592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:25.029546 12592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:25.031054 12592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:25.031188 12592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:25.031378 12592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:25.031629 12592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:25.031677 12592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:25.031713 12592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:25.031737 12592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:25.036943 12592 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:36163
I20260809 23:14:25.037009 12816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:36163 every 8 connection(s)
I20260809 23:14:25.037266 12592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:14:25 dist-test-slave-7vjv krb5kdc[11941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317265, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:14:25.045004 12085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:25.037919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:33589 (local address 127.2.76.254:40489)
0809 23:14:25.037990 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:25.037992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:25.038125 (+ 133us) server_negotiation.cc:407] Connection header received
0809 23:14:25.038817 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:25.038818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:25.038827 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:25.038872 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:25.039325 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:25.039746 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:25.040628 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:25.040782 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:25.040830 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:25.042657 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:25.042658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:25.042658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:25.042664 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:25.043861 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:25.044290 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:25.044291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:25.044291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:25.044345 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:25.044550 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:25.044550 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:25.044551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:25.044650 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:25.044694 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:25.044889 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 23:14:25.044948 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 23:14:25.045137 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:25.037868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40489 (local address 127.2.76.195:33589)
0809 23:14:25.038015 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:25.038045 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:25.038741 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:25.038915 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:25.038918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:25.039117 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:25.039263 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:25.039266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:25.039820 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:25.039821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:25.040562 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:25.040564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:25.040848 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:25.040848 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:25.040927 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:25.041357 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:25.041371 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:25.042577 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:25.043937 (+ 1360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:25.043944 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:25.043945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:25.044227 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:25.044404 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:25.044405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:25.044405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:25.044508 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:25.044699 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:25.044715 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:25.044805 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 23:14:25.044871 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 23:14:25.044971 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260809 23:14:25.045603 12817 heartbeater.cc:347] Connected to a master server at 127.2.76.254:40489
I20260809 23:14:25.045682 12817 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:25.045796 12817 heartbeater.cc:511] Master 127.2.76.254:40489 requested a full tablet report, sending...
I20260809 23:14:25.046181 12016 ts_manager.cc:194] Registered new tserver with Master: 87d1ddb3450543768b3812b714cf7fa1 (127.2.76.195:36163)
I20260809 23:14:25.046520 12016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:33589
I20260809 23:14:25.046657 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 12592
I20260809 23:14:25.046795 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:14:25.047006 2355 external_mini_cluster.cc:1428] Setting key 879d36fda81b1fcba8fec1ec92796df9
I20260809 23:14:25.047945 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/test-admin.keytab.
Aug 09 23:14:25 dist-test-slave-7vjv krb5kdc[11941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:25.063711 2355 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 23:14:25.065523 12830 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:14:25.064652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35588 (local address 127.2.76.193:42479)
0809 23:14:25.064859 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:25.064863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:25.064879 (+ 16us) server_negotiation.cc:407] Connection header received
0809 23:14:25.064901 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:25.064903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:25.064915 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:25.064990 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:25.065439 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35588: BlockingRecv error: recv got EOF from 127.0.0.1:35588 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":111,"threads_started":1}
W20260809 23:14:25.065716 2355 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/krb5cc' of type FILE
Aug 09 23:14:25 dist-test-slave-7vjv krb5kdc[11941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:25.066707 2355 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 23:14:25 dist-test-slave-7vjv krb5kdc[11941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:25.067598 2355 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 23:14:25 dist-test-slave-7vjv krb5kdc[11941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.193@KRBTEST.COM
I20260809 23:14:25.074294 12830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:25.068027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35604 (local address 127.2.76.193:42479)
0809 23:14:25.068082 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:25.068084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:25.068148 (+ 64us) server_negotiation.cc:407] Connection header received
0809 23:14:25.068166 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:25.068167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:25.068177 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:25.068210 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:25.068531 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:25.069252 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:25.069931 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:25.070226 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:25.070274 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:25.072000 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:25.072007 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:25.072007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:25.072013 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:25.073314 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:25.073613 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:25.073615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:25.073615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:25.073665 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:25.073809 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:25.073810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:25.073810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:25.073915 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:25.074048 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:25.074134 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 23:14:25.074189 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 23:14:25.075989 12830 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:14:25.075313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35610 (local address 127.2.76.193:42479)
0809 23:14:25.075368 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:25.075370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:25.075510 (+ 140us) server_negotiation.cc:407] Connection header received
0809 23:14:25.075605 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:25.075606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:25.075628 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:25.075684 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:25.075946 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35610: BlockingRecv error: recv got EOF from 127.0.0.1:35610 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 23:14:25.076076 2355 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/krb5cc' of type FILE
Aug 09 23:14:25 dist-test-slave-7vjv krb5kdc[11941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:25.076974 2355 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 23:14:25 dist-test-slave-7vjv krb5kdc[11941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:25.077694 2355 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 23:14:25 dist-test-slave-7vjv krb5kdc[11941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.193@KRBTEST.COM
I20260809 23:14:25.083853 12830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:25.078105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35612 (local address 127.2.76.193:42479)
0809 23:14:25.078167 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:25.078168 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:25.078199 (+ 31us) server_negotiation.cc:407] Connection header received
0809 23:14:25.078264 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:25.078264 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:25.078272 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:25.078329 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:25.078714 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:25.079179 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:25.079866 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:25.080072 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:25.080162 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:25.081818 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:25.081819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:25.081819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:25.081826 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:25.082790 (+ 964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:25.083096 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:25.083097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:25.083097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:25.083147 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:25.083404 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:25.083405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:25.083405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:25.083496 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:25.083588 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:25.083698 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 23:14:25.083771 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 23:14:25.085383 12830 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:14:25.084802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35624 (local address 127.2.76.193:42479)
0809 23:14:25.084858 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:25.084860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:25.084882 (+ 22us) server_negotiation.cc:407] Connection header received
0809 23:14:25.084959 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:25.084960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:25.084968 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:25.085017 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:25.085346 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35624: BlockingRecv error: recv got EOF from 127.0.0.1:35624 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 23:14:25.085466 2355 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786317238298104-2355-0/krb5kdc/krb5cc' of type FILE
Aug 09 23:14:25 dist-test-slave-7vjv krb5kdc[11941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:25.086397 2355 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 23:14:25 dist-test-slave-7vjv krb5kdc[11941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:25.087292 2355 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 23:14:25 dist-test-slave-7vjv krb5kdc[11941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.193@KRBTEST.COM
I20260809 23:14:25.093014 12830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:25.087734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35628 (local address 127.2.76.193:42479)
0809 23:14:25.087783 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:25.087784 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:25.087810 (+ 26us) server_negotiation.cc:407] Connection header received
0809 23:14:25.087852 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:25.087852 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:25.087858 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:25.087902 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:25.088254 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:25.088648 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:25.089229 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:25.089392 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:25.089435 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:25.091107 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:25.091108 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:25.091108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:25.091114 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:25.092039 (+ 925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:25.092335 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:25.092335 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:25.092335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:25.092378 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:25.092563 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:25.092563 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:25.092564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:25.092633 (+ 69us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:25.092689 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:25.092851 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 23:14:25.092919 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 23:14:25.093598 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 12122
I20260809 23:14:25.106001 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 12361
I20260809 23:14:25.112800 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 12592
I20260809 23:14:25.128258 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 11961
2026-08-09T23:14:25Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1905 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:14:25 dist-test-slave-7vjv krb5kdc[12852](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:14:25 dist-test-slave-7vjv krb5kdc[12852](info): set up 2 sockets
Aug 09 23:14:25 dist-test-slave-7vjv krb5kdc[12852](info): commencing operation
krb5kdc: starting...
W20260809 23:14:26.293975 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:14:26 dist-test-slave-7vjv krb5kdc[12852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:14:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:26Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.254.keytab.
Entry for principal oryx/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.254.keytab.
I20260809 23:14:26.404254 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:37751
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.254.keytab
--principal=oryx/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:37751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:26.476321 12872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:26.476493 12872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:26.476519 12872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:26.477530 12872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:26.477566 12872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:26.477578 12872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:26.477624 12872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:26.477638 12872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:14:26.478888 12872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:37751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.254.keytab
--principal=oryx/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:37751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:26.479130 12872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:26.479329 12872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:26.483856 12887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:26.483844 12883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:26.483836 12881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:26.483875 12872 server_base.cc:1034] running on GCE node
I20260809 23:14:26.484268 12872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:26.484549 12872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:26.485713 12872 hybrid_clock.cc:648] HybridClock initialized: now 1786317266485693 us; error 31 us; skew 500 ppm
Aug 09 23:14:26 dist-test-slave-7vjv krb5kdc[12852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317266, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:26.487963 12872 init.cc:377] Logged in from keytab as oryx/127.2.76.254@KRBTEST.COM (short username oryx)
I20260809 23:14:26.488394 12872 webserver.cc:460] Webserver started at http://127.2.76.254:45777/ using document root <none> and password file <none>
I20260809 23:14:26.488606 12872 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:26.488672 12872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:26.488801 12872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:26.489823 12872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "60c9c82a25364f28818b186daee44f4c"
format_stamp: "Formatted at 2026-08-09 23:14:26 on dist-test-slave-7vjv"
server_key: "696cc6309336c0d7d9bc6e8dd5a12611"
server_key_iv: "c37547756581bb55f880b4e8b1366336"
server_key_version: "encryptionkey@0"
I20260809 23:14:26.490213 12872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "60c9c82a25364f28818b186daee44f4c"
format_stamp: "Formatted at 2026-08-09 23:14:26 on dist-test-slave-7vjv"
server_key: "696cc6309336c0d7d9bc6e8dd5a12611"
server_key_iv: "c37547756581bb55f880b4e8b1366336"
server_key_version: "encryptionkey@0"
I20260809 23:14:26.491410 12872 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:14:26.492130 12899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:26.492303 12872 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:26.492398 12872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "60c9c82a25364f28818b186daee44f4c"
format_stamp: "Formatted at 2026-08-09 23:14:26 on dist-test-slave-7vjv"
server_key: "696cc6309336c0d7d9bc6e8dd5a12611"
server_key_iv: "c37547756581bb55f880b4e8b1366336"
server_key_version: "encryptionkey@0"
I20260809 23:14:26.492478 12872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:26.511143 12872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:26.512960 12872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:26.513114 12872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:26.516705 12872 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:37751
I20260809 23:14:26.516765 12989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:37751 every 8 connection(s)
I20260809 23:14:26.517056 12872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:26.517946 12993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:26.518353 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 12872
I20260809 23:14:26.518492 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:26.518659 2355 external_mini_cluster.cc:1428] Setting key 4346ec1ab91ceafdf39644a7ff8b0c3b
I20260809 23:14:26.520721 12993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c: Bootstrap starting.
I20260809 23:14:26.521337 12993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:26.521638 12993 log.cc:824] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:26.522253 12993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c: No bootstrap required, opened a new log
I20260809 23:14:26.523957 12993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } }
I20260809 23:14:26.524086 12993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:26.524221 12993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c9c82a25364f28818b186daee44f4c, State: Initialized, Role: FOLLOWER
I20260809 23:14:26.524356 12993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } }
I20260809 23:14:26.524456 12993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:26.524505 12993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:26.524551 12993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:26.525027 12993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } }
I20260809 23:14:26.525142 12993 leader_election.cc:302] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60c9c82a25364f28818b186daee44f4c; no voters:
Aug 09 23:14:26 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
I20260809 23:14:26.525310 12993 leader_election.cc:288] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:26.525358 13002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:26.525516 13002 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [term 1 LEADER]: Becoming Leader. State: Replica: 60c9c82a25364f28818b186daee44f4c, State: Running, Role: LEADER
I20260809 23:14:26.525604 12993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:26.525638 13002 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } }
I20260809 23:14:26.526264 13004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60c9c82a25364f28818b186daee44f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } }
I20260809 23:14:26.526345 13004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:26.526822 13006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60c9c82a25364f28818b186daee44f4c. Latest consensus state: current_term: 1 leader_uuid: "60c9c82a25364f28818b186daee44f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } }
I20260809 23:14:26.526909 13006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:26.527041 13020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:26.527751 13020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:26.529330 13020 catalog_manager.cc:1347] Generated new cluster ID: d4c48d90d186463aa8ea2ca835a219ea
I20260809 23:14:26.529390 13020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:14:26.529536 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:26.519132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37644 (local address 127.2.76.254:37751)
0809 23:14:26.519301 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:26.519309 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:26.519338 (+ 29us) server_negotiation.cc:407] Connection header received
0809 23:14:26.519815 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:26.519822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:26.519880 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:26.520052 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:26.520635 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:26.521895 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:26.522669 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:26.523108 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:26.523226 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:26.525758 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:26.525762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:26.525762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:26.525774 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:26.527866 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:26.528277 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:26.528280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:26.528280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:26.528384 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:26.528584 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:26.528585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:26.528585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:26.528817 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:26.528964 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:26.529200 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 23:14:26.529320 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260809 23:14:26.542405 13020 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:26.542963 13020 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:14:26.549154 13020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c: Generated new TSK 0
I20260809 23:14:26.549289 13020 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.193.keytab.
Entry for principal oryx/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.193.keytab.
I20260809 23:14:26.580698 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37751
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.193.keytab
--principal=oryx/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:26.655078 13040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:26.655277 13040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:26.655301 13040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:26.656371 13040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:26.656414 13040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:26.656468 13040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:14:26.657824 13040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.193.keytab
--principal=oryx/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:26.658059 13040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:26.658265 13040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:26.663002 13054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:26.663024 13050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:26.663022 13051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:26.663055 13040 server_base.cc:1034] running on GCE node
I20260809 23:14:26.663457 13040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:26.663713 13040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:26.664880 13040 hybrid_clock.cc:648] HybridClock initialized: now 1786317266664850 us; error 39 us; skew 500 ppm
Aug 09 23:14:26 dist-test-slave-7vjv krb5kdc[12852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317266, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:26.667147 13040 init.cc:377] Logged in from keytab as oryx/127.2.76.193@KRBTEST.COM (short username oryx)
I20260809 23:14:26.667526 13040 webserver.cc:460] Webserver started at http://127.2.76.193:33125/ using document root <none> and password file <none>
I20260809 23:14:26.667742 13040 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:26.667804 13040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:26.667943 13040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:26.668895 13040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "66402c0016014fbbbc0e6c094ea5b551"
format_stamp: "Formatted at 2026-08-09 23:14:26 on dist-test-slave-7vjv"
server_key: "9a0b6a68cf25454703d5afed7cb1fca9"
server_key_iv: "7dbe443cb6f167db53505e5e49dbf299"
server_key_version: "encryptionkey@0"
I20260809 23:14:26.669237 13040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "66402c0016014fbbbc0e6c094ea5b551"
format_stamp: "Formatted at 2026-08-09 23:14:26 on dist-test-slave-7vjv"
server_key: "9a0b6a68cf25454703d5afed7cb1fca9"
server_key_iv: "7dbe443cb6f167db53505e5e49dbf299"
server_key_version: "encryptionkey@0"
I20260809 23:14:26.670550 13040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:14:26.671229 13064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:26.671386 13040 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:26.671468 13040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "66402c0016014fbbbc0e6c094ea5b551"
format_stamp: "Formatted at 2026-08-09 23:14:26 on dist-test-slave-7vjv"
server_key: "9a0b6a68cf25454703d5afed7cb1fca9"
server_key_iv: "7dbe443cb6f167db53505e5e49dbf299"
server_key_version: "encryptionkey@0"
I20260809 23:14:26.671550 13040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:26.692353 13040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:26.694594 13040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:26.694864 13040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:26.695111 13040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:26.695402 13040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:26.695452 13040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:26.695487 13040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:26.695520 13040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:26.700984 13040 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:40739
I20260809 23:14:26.701037 13267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:40739 every 8 connection(s)
I20260809 23:14:26.701308 13040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:14:26.705058 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 13040
I20260809 23:14:26.705132 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:14:26.705312 2355 external_mini_cluster.cc:1428] Setting key b0214042e50f6f6d29ff85c7569bd683
Aug 09 23:14:26 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317266, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
I20260809 23:14:26.709791 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:26.701854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:55413)
0809 23:14:26.701996 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:26.702038 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:26.702472 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:26.702692 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:26.702695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:26.703170 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:26.703325 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:26.703329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:26.703901 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:26.703902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:26.704678 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:26.704681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:26.705031 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:26.705032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:26.705183 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:26.705658 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:26.705667 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:26.706952 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:26.708647 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:26.708653 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:26.708654 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:26.708866 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:26.709073 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:26.709073 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:26.709074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:26.709170 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:26.709415 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:26.709419 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:26.709505 (+ 86us) client_negotiation.cc:769] Sending connection context
0809 23:14:26.709575 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 23:14:26.709669 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
I20260809 23:14:26.709820 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:26.701899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:55413 (local address 127.2.76.254:37751)
0809 23:14:26.701941 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:26.701943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:26.702123 (+ 180us) server_negotiation.cc:407] Connection header received
0809 23:14:26.702560 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:26.702561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:26.702570 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:26.702622 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:26.703399 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:26.703831 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:26.704752 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:26.704933 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:26.705017 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:26.707045 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:26.707046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:26.707046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:26.707054 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:26.708549 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:26.708939 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:26.708940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:26.708940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:26.709009 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:26.709229 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:26.709230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:26.709230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:26.709358 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:26.709420 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:26.709619 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 23:14:26.709701 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:14:26.710515 13268 heartbeater.cc:347] Connected to a master server at 127.2.76.254:37751
I20260809 23:14:26.710614 13268 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:26.710784 13268 heartbeater.cc:511] Master 127.2.76.254:37751 requested a full tablet report, sending...
I20260809 23:14:26.711236 12933 ts_manager.cc:194] Registered new tserver with Master: 66402c0016014fbbbc0e6c094ea5b551 (127.2.76.193:40739)
I20260809 23:14:26.711757 12933 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:55413
WARNING: no policy specified for oryx/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.194.keytab.
Entry for principal oryx/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.194.keytab.
I20260809 23:14:26.728778 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37751
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.194.keytab
--principal=oryx/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:26.801581 13278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:26.801759 13278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:26.801795 13278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:26.802754 13278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:26.802820 13278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:26.802855 13278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:14:26.804070 13278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.194.keytab
--principal=oryx/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:26.804360 13278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:26.804566 13278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:26.808745 13289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:26.808805 13290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:26.808754 13295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:26.809163 13278 server_base.cc:1034] running on GCE node
I20260809 23:14:26.809379 13278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:26.809588 13278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:26.810738 13278 hybrid_clock.cc:648] HybridClock initialized: now 1786317266810722 us; error 33 us; skew 500 ppm
Aug 09 23:14:26 dist-test-slave-7vjv krb5kdc[12852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317266, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:26.812994 13278 init.cc:377] Logged in from keytab as oryx/127.2.76.194@KRBTEST.COM (short username oryx)
I20260809 23:14:26.813393 13278 webserver.cc:460] Webserver started at http://127.2.76.194:34447/ using document root <none> and password file <none>
I20260809 23:14:26.813603 13278 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:26.813666 13278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:26.813802 13278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:26.814663 13278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "3d67646499fc4284afd0d289a2c9c04f"
format_stamp: "Formatted at 2026-08-09 23:14:26 on dist-test-slave-7vjv"
server_key: "f7c3366bc3575b5785778a5bcbd667aa"
server_key_iv: "5cb934708b4cc014dd0a5619f440a119"
server_key_version: "encryptionkey@0"
I20260809 23:14:26.814987 13278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "3d67646499fc4284afd0d289a2c9c04f"
format_stamp: "Formatted at 2026-08-09 23:14:26 on dist-test-slave-7vjv"
server_key: "f7c3366bc3575b5785778a5bcbd667aa"
server_key_iv: "5cb934708b4cc014dd0a5619f440a119"
server_key_version: "encryptionkey@0"
I20260809 23:14:26.816169 13278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:14:26.816869 13306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:26.817032 13278 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:26.817127 13278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "3d67646499fc4284afd0d289a2c9c04f"
format_stamp: "Formatted at 2026-08-09 23:14:26 on dist-test-slave-7vjv"
server_key: "f7c3366bc3575b5785778a5bcbd667aa"
server_key_iv: "5cb934708b4cc014dd0a5619f440a119"
server_key_version: "encryptionkey@0"
I20260809 23:14:26.817209 13278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:26.825410 13278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:26.827103 13278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:26.827232 13278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:26.827446 13278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:26.827692 13278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:26.827739 13278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:26.827772 13278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:26.827796 13278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:26.833256 13278 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:34053
I20260809 23:14:26.833300 13505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:34053 every 8 connection(s)
I20260809 23:14:26.833534 13278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:14:26 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317266, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
I20260809 23:14:26.841454 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:26.834056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:50447 (local address 127.2.76.254:37751)
0809 23:14:26.834118 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:26.834120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:26.834273 (+ 153us) server_negotiation.cc:407] Connection header received
0809 23:14:26.834620 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:26.834621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:26.834630 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:26.834695 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:26.835456 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:26.835931 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:26.836899 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:26.837103 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:26.837151 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:26.839109 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:26.839110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:26.839110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:26.839117 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:26.840377 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:26.840723 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:26.840724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:26.840724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:26.840775 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:26.840973 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:26.840973 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:26.840974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:26.841073 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:26.841123 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:26.841327 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 23:14:26.841397 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:14:26.841569 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:26.834028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:50447)
0809 23:14:26.834169 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:26.834197 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:26.834550 (+ 353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:26.834737 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:26.834740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:26.835222 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:26.835389 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:26.835393 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:26.836010 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:26.836011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:26.836829 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:26.836831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:26.837171 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:26.837171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:26.837265 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:26.837700 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:26.837707 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:26.838900 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:26.840448 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:26.840454 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:26.840455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:26.840658 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:26.840831 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:26.840832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:26.840832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:26.840921 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:26.841128 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:26.841132 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:26.841226 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 23:14:26.841299 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:14:26.841409 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":88,"threads_started":1}
I20260809 23:14:26.842062 13507 heartbeater.cc:347] Connected to a master server at 127.2.76.254:37751
I20260809 23:14:26.842131 13507 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:26.842252 13507 heartbeater.cc:511] Master 127.2.76.254:37751 requested a full tablet report, sending...
I20260809 23:14:26.842483 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 13278
I20260809 23:14:26.842585 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:14:26.842617 12933 ts_manager.cc:194] Registered new tserver with Master: 3d67646499fc4284afd0d289a2c9c04f (127.2.76.194:34053)
I20260809 23:14:26.842728 2355 external_mini_cluster.cc:1428] Setting key dde91c41e97d717daf5da071e1fc4d80
I20260809 23:14:26.843026 12933 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:50447
WARNING: no policy specified for oryx/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.195.keytab.
Entry for principal oryx/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.195.keytab.
I20260809 23:14:26.866523 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37751
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.195.keytab
--principal=oryx/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:26.937214 13524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:26.937363 13524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:26.937382 13524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:26.938345 13524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:26.938381 13524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:26.938428 13524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:14:26.939733 13524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.195.keytab
--principal=oryx/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:26.939934 13524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:26.940169 13524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:26.944497 13533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:26.944630 13524 server_base.cc:1034] running on GCE node
W20260809 23:14:26.944542 13536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:26.944515 13531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:26.944875 13524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:26.945096 13524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:26.946249 13524 hybrid_clock.cc:648] HybridClock initialized: now 1786317266946235 us; error 40 us; skew 500 ppm
Aug 09 23:14:26 dist-test-slave-7vjv krb5kdc[12852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317266, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:26.948562 13524 init.cc:377] Logged in from keytab as oryx/127.2.76.195@KRBTEST.COM (short username oryx)
I20260809 23:14:26.948971 13524 webserver.cc:460] Webserver started at http://127.2.76.195:43159/ using document root <none> and password file <none>
I20260809 23:14:26.949165 13524 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:26.949222 13524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:26.949337 13524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:26.950172 13524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "fbd85ed7dd8d47058f5e51279d68ed39"
format_stamp: "Formatted at 2026-08-09 23:14:26 on dist-test-slave-7vjv"
server_key: "ea15635f084f623cde1a10279f0f82c0"
server_key_iv: "d16448d655673f52ba461dccc5b14f41"
server_key_version: "encryptionkey@0"
I20260809 23:14:26.950485 13524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "fbd85ed7dd8d47058f5e51279d68ed39"
format_stamp: "Formatted at 2026-08-09 23:14:26 on dist-test-slave-7vjv"
server_key: "ea15635f084f623cde1a10279f0f82c0"
server_key_iv: "d16448d655673f52ba461dccc5b14f41"
server_key_version: "encryptionkey@0"
I20260809 23:14:26.951624 13524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:14:26.952275 13549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:26.952435 13524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:26.952521 13524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "fbd85ed7dd8d47058f5e51279d68ed39"
format_stamp: "Formatted at 2026-08-09 23:14:26 on dist-test-slave-7vjv"
server_key: "ea15635f084f623cde1a10279f0f82c0"
server_key_iv: "d16448d655673f52ba461dccc5b14f41"
server_key_version: "encryptionkey@0"
I20260809 23:14:26.952598 13524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:26.961046 13524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:26.962523 13524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:26.962663 13524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:26.962852 13524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:26.963109 13524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:26.963156 13524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:26.963191 13524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:26.963213 13524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:26.968398 13524 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:33227
I20260809 23:14:26.968430 13752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:33227 every 8 connection(s)
I20260809 23:14:26.968688 13524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:14:26.970263 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 13524
I20260809 23:14:26.970336 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:14:26.970500 2355 external_mini_cluster.cc:1428] Setting key c03f497522654816f4303a0db525a8ea
Aug 09 23:14:26 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317266, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
I20260809 23:14:26.976374 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:26.969346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:58881 (local address 127.2.76.254:37751)
0809 23:14:26.969392 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:26.969394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:26.969547 (+ 153us) server_negotiation.cc:407] Connection header received
0809 23:14:26.969881 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:26.969882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:26.969891 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:26.969933 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:26.970784 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:26.971424 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:26.972279 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:26.972441 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:26.972487 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:26.974251 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:26.974252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:26.974253 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:26.974258 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:26.975319 (+ 1061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:26.975649 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:26.975649 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:26.975650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:26.975694 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:26.975872 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:26.975873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:26.975873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:26.975954 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:26.975998 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:26.976249 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 23:14:26.976302 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:14:26.976563 13756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:26.969322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:58881)
0809 23:14:26.969444 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:26.969470 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:26.969813 (+ 343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:26.969973 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:26.969976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:26.970475 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:26.970705 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:26.970711 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:26.971497 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:26.971498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:26.972218 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:26.972220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:26.972508 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:26.972509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:26.972589 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:26.972996 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:26.973003 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:26.974168 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:26.975393 (+ 1225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:26.975398 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:26.975400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:26.975588 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:26.975745 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:26.975746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:26.975746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:26.975827 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:26.976000 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:26.976005 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:26.976143 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 23:14:26.976230 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 23:14:26.976353 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":71,"threads_started":1}
I20260809 23:14:26.977000 13753 heartbeater.cc:347] Connected to a master server at 127.2.76.254:37751
I20260809 23:14:26.977073 13753 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:26.977183 13753 heartbeater.cc:511] Master 127.2.76.254:37751 requested a full tablet report, sending...
I20260809 23:14:26.977487 12928 ts_manager.cc:194] Registered new tserver with Master: fbd85ed7dd8d47058f5e51279d68ed39 (127.2.76.195:33227)
I20260809 23:14:26.977775 12928 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:58881
I20260809 23:14:26.982016 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.76.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.76.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.76.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.253.keytab.
Entry for principal oryx/127.2.76.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.253.keytab.
I20260809 23:14:27.004781 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.253:40323
--webserver_interface=127.2.76.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.253.keytab
--principal=oryx/127.2.76.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:37751,127.2.76.253:40323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:27.078573 13760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:27.078765 13760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:27.078800 13760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:27.079803 13760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:27.079864 13760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:27.079886 13760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:27.079912 13760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:27.079936 13760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:14:27.081456 13760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:37751,127.2.76.253:40323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.253.keytab
--principal=oryx/127.2.76.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.253:40323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.76.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
W20260809 23:14:27.081615 13760 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.76.254:37751,127.2.76.253:40323'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 23:14:27.081766 13760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:27.081969 13760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:27.086220 13778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:27.086199 13771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:27.086206 13774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:27.086287 13760 server_base.cc:1034] running on GCE node
I20260809 23:14:27.086728 13760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:27.086984 13760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:27.088150 13760 hybrid_clock.cc:648] HybridClock initialized: now 1786317267088133 us; error 27 us; skew 500 ppm
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:27.090291 13760 init.cc:377] Logged in from keytab as oryx/127.2.76.253@KRBTEST.COM (short username oryx)
I20260809 23:14:27.090720 13760 webserver.cc:460] Webserver started at http://127.2.76.253:45751/ using document root <none> and password file <none>
I20260809 23:14:27.090936 13760 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:27.090996 13760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:27.091146 13760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:27.091995 13760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/data/instance:
uuid: "ea2558e1baef47b2b24b355d6270d1e3"
format_stamp: "Formatted at 2026-08-09 23:14:27 on dist-test-slave-7vjv"
server_key: "25fbf3a9a84f2e54c6ebb89cd10776dc"
server_key_iv: "f7b130dc349752c97ba6e6d60d57633c"
server_key_version: "encryptionkey@0"
I20260809 23:14:27.092351 13760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/wal/instance:
uuid: "ea2558e1baef47b2b24b355d6270d1e3"
format_stamp: "Formatted at 2026-08-09 23:14:27 on dist-test-slave-7vjv"
server_key: "25fbf3a9a84f2e54c6ebb89cd10776dc"
server_key_iv: "f7b130dc349752c97ba6e6d60d57633c"
server_key_version: "encryptionkey@0"
I20260809 23:14:27.093464 13760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:27.094108 13789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.094274 13760 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:27.094363 13760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/wal
uuid: "ea2558e1baef47b2b24b355d6270d1e3"
format_stamp: "Formatted at 2026-08-09 23:14:27 on dist-test-slave-7vjv"
server_key: "25fbf3a9a84f2e54c6ebb89cd10776dc"
server_key_iv: "f7b130dc349752c97ba6e6d60d57633c"
server_key_version: "encryptionkey@0"
I20260809 23:14:27.094444 13760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:27.121268 13760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:27.123548 13760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:27.123680 13760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:27.127893 13760 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.253:40323
I20260809 23:14:27.127923 13876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.253:40323 every 8 connection(s)
I20260809 23:14:27.128243 13760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/data/info.pb
I20260809 23:14:27.129145 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 13760
I20260809 23:14:27.129230 13877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:27.131201 13877 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.76.253@KRBTEST.COM
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.253@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
I20260809 23:14:27.138864 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.131818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37656 (local address 127.2.76.254:37751)
0809 23:14:27.131862 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.131864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.131979 (+ 115us) server_negotiation.cc:407] Connection header received
0809 23:14:27.132053 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.132054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.132062 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.132161 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.132645 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.133268 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.134299 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.134604 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.134685 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.136612 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.136613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.136614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.136620 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.137720 (+ 1100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.138064 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.138064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.138065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.138110 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.138335 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.138336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.138336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.138460 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.138551 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.138716 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.138784 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:14:27.139042 13884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.131736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:37656)
0809 23:14:27.131902 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.131934 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.132001 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.132183 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.132185 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.132516 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.132574 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.132576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.133342 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.133343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.134221 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.134224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.134689 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.134690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.134742 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.135177 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.135184 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.136533 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.137811 (+ 1278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.137817 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.137818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.137995 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.138178 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.138178 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.138179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.138277 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.138545 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.138549 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.138602 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.138682 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.138851 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":60,"thread_start_us":90,"threads_started":1}
I20260809 23:14:27.139042 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.129750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59230 (local address 127.2.76.253:40323)
0809 23:14:27.129926 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.129932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.129953 (+ 21us) server_negotiation.cc:407] Connection header received
0809 23:14:27.130408 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.130413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.130452 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.130551 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.131260 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.132341 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.133206 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.133696 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.133793 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.136180 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.136186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.136186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.136192 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.137711 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.138029 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.138032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.138032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.138114 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.138284 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.138285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.138285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.138465 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.138609 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.138673 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.138790 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260809 23:14:27.139742 13877 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.253@KRBTEST.COM for oryx/127.2.76.253@KRBTEST.COM
I20260809 23:14:27.144918 13884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.139953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:40323 (local address 127.0.0.1:59246)
0809 23:14:27.139999 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.140008 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.140042 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.140170 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.140170 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.140277 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.140318 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.140319 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.140826 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.140827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.141209 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.141210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.141548 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.141548 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.141591 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.141878 (+ 287us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.141882 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.142838 (+ 956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.144279 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.144280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.144280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.144419 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.144554 (+ 135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.144554 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.144555 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.144595 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.144727 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.144727 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.144754 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.144793 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.144854 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 23:14:27.144968 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.140004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59246 (local address 127.2.76.253:40323)
0809 23:14:27.140051 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.140053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.140059 (+ 6us) server_negotiation.cc:407] Connection header received
0809 23:14:27.140075 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.140076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.140082 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.140158 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.140370 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.140750 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.141267 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.141483 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.141542 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.142908 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.142909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.142909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.142914 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.144219 (+ 1305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.144469 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.144469 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.144469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.144510 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.144626 (+ 116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.144626 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.144626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.144696 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.144734 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.144806 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.144891 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":32}
I20260809 23:14:27.145946 13877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: Bootstrap starting.
I20260809 23:14:27.146694 13877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:27.146966 13877 log.cc:824] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:27.147557 13877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: No bootstrap required, opened a new log
I20260809 23:14:27.148872 13877 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } }
I20260809 23:14:27.149003 13877 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:27.149051 13877 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea2558e1baef47b2b24b355d6270d1e3, State: Initialized, Role: FOLLOWER
I20260809 23:14:27.149174 13877 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } }
I20260809 23:14:27.149425 13877 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:27.149394 13887 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } } }
I20260809 23:14:27.149492 13887 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 23:14:27.150899 13910 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:14:27.150969 13910 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:14:27.152563 13760 master_runner.cc:428] Detected that this master ea2558e1baef47b2b24b355d6270d1e3 is joining an existing cluster
I20260809 23:14:27.152631 13760 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.76.253:40323
I20260809 23:14:27.159901 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.153542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.2.76.254:37751)
0809 23:14:27.153578 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.153580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.153745 (+ 165us) server_negotiation.cc:407] Connection header received
0809 23:14:27.153861 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.153862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.153873 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.153927 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.154310 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.154964 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.155776 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.156025 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.156078 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.157157 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.157158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.157159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.157165 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.158645 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.159029 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.159030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.159030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.159083 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.159301 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.159302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.159302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.159419 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.159467 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.159732 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.159815 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:14:27.159906 13924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.153537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:37662)
0809 23:14:27.153695 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.153703 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.153756 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.153967 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.153968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.154192 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.154246 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.154248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.155030 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.155031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.155694 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.155702 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.156160 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.156161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.156215 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.156647 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.156652 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.157062 (+ 410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.158736 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.158737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.158738 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.158938 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.159156 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.159157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.159157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.159219 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.159542 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.159543 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.159596 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.159692 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.159788 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":79,"threads_started":1}
I20260809 23:14:27.159966 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.153624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59248 (local address 127.2.76.253:40323)
0809 23:14:27.153664 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.153666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.153843 (+ 177us) server_negotiation.cc:407] Connection header received
0809 23:14:27.153897 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.153898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.153905 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.153952 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.154262 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.154861 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.155942 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.156229 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.156296 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.157307 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.157308 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.157308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.157314 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.158814 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.159198 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.159199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.159199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.159243 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.159424 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.159425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.159426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.159552 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.159637 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.159777 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.159841 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:14:27.160293 13926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.153681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:40323 (local address 127.0.0.1:59248)
0809 23:14:27.153796 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.153805 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.153858 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.153957 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.153957 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.154127 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.154193 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.154194 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.154948 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.154949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.155856 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.155860 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.156307 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.156309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.156375 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.156816 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.156820 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.157224 (+ 404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.158882 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.158882 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.158883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.159122 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.159309 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.159310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.159310 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.159360 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.159630 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.159631 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.159669 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.159924 (+ 255us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.160194 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317266, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.254@KRBTEST.COM for oryx/127.2.76.253@KRBTEST.COM
I20260809 23:14:27.168861 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.162175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:40323 (local address 127.0.0.1:59254)
0809 23:14:27.162329 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.162364 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.162427 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.162577 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.162578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.162786 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.162825 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.162826 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.163616 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.163616 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.164199 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.164201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.164626 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.164627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.164676 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.165091 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.165095 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.166406 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.167885 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.167892 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.167893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.168087 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.168315 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.168315 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.168316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.168371 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.168615 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.168621 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.168659 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.168720 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.168800 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
I20260809 23:14:27.168921 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.162243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59254 (local address 127.2.76.253:40323)
0809 23:14:27.162283 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.162285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.162433 (+ 148us) server_negotiation.cc:407] Connection header received
0809 23:14:27.162485 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.162486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.162495 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.162575 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.162892 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.163541 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.164270 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.164549 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.164611 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.166492 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.166493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.166494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.166505 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.167809 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.168188 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.168189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.168189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.168247 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.168431 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.168432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.168432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.168553 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.168619 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.168765 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.168839 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:14:27.169476 12928 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.76.253:40323: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3"
member_type: NON_VOTER
last_known_addr {
host: "127.2.76.253"
port: 40323
}
attrs {
promote: true
}
}
dest_uuid: "60c9c82a25364f28818b186daee44f4c"
cas_config_opid_index: -1
I20260809 23:14:27.169818 12928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: NON_VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: true } }
I20260809 23:14:27.170334 13933 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ea2558e1baef47b2b24b355d6270d1e3 (127.2.76.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: NON_VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: true } } }
I20260809 23:14:27.170630 13835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:27.171470 13933 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.76.253:40323
I20260809 23:14:27.171540 13835 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Refusing update from remote peer 60c9c82a25364f28818b186daee44f4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 23:14:27.171701 13934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "60c9c82a25364f28818b186daee44f4c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: NON_VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: true } } }
I20260809 23:14:27.171798 13934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:27.171794 13760 master.cc:537] Master@127.2.76.253:40323 shutting down...
I20260809 23:14:27.172559 13934 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: NON_VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 23:14:27.173302 12909 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 23:14:27.173723 12909 consensus_peers.cc:487] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c -> Peer ea2558e1baef47b2b24b355d6270d1e3 (127.2.76.253:40323): Couldn't send request to peer ea2558e1baef47b2b24b355d6270d1e3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 23:14:27.174371 13760 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:27.174469 13760 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:27.174525 13760 tablet_replica.cc:330] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: stopping tablet replica
I20260809 23:14:27.186589 13760 master.cc:552] Master@127.2.76.253:40323 shutdown complete.
I20260809 23:14:27.186625 13760 master_runner.cc:305] Clearing existing system tablet
I20260809 23:14:27.186748 13760 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:27.186784 13760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:27.187314 13760 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:14:27.187763 13940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.187865 13760 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:27.187914 13760 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.187933 13760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/wal
uuid: "ea2558e1baef47b2b24b355d6270d1e3"
format_stamp: "Formatted at 2026-08-09 23:14:27 on dist-test-slave-7vjv"
server_key: "25fbf3a9a84f2e54c6ebb89cd10776dc"
server_key_iv: "f7b130dc349752c97ba6e6d60d57633c"
server_key_version: "encryptionkey@0"
I20260809 23:14:27.188925 13760 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:27.190095 13760 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 23:14:27.190194 13760 log.cc:1196] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 23:14:27.190397 13760 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: Deleting consensus metadata
I20260809 23:14:27.190519 13760 master_runner.cc:315] Copying system tablet from 127.2.76.254:37751
I20260809 23:14:27.191071 13760 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: tablet copy: Beginning tablet copy session from remote peer at address 127.2.76.254:37751
I20260809 23:14:27.196729 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.191419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37678 (local address 127.2.76.254:37751)
0809 23:14:27.191456 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.191458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.191522 (+ 64us) server_negotiation.cc:407] Connection header received
0809 23:14:27.191583 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.191584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.191591 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.191629 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.191940 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.192422 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.193424 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.193581 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.193624 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.194453 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.194454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.194454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.194459 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.195752 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.196188 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.196189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.196189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.196231 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.196403 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.196404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.196404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.196473 (+ 69us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.196514 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.196640 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.196686 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:14:27.196894 13953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.191342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:37678)
0809 23:14:27.191461 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.191471 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.191530 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.191643 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.191644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.191823 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.191877 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.191879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.192506 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.192507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.193354 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.193356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.193645 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.193646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.193710 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.194041 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.194046 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.194368 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.195821 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.195822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.195822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.196054 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.196294 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.196295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.196295 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.196353 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.196523 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.196523 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.196562 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.196628 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.196711 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":77,"threads_started":1}
I20260809 23:14:27.197238 12970 tablet_copy_service.cc:140] P 60c9c82a25364f28818b186daee44f4c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ea2558e1baef47b2b24b355d6270d1e3 ({username='oryx', principal='oryx/127.2.76.253@KRBTEST.COM'} at 127.0.0.1:37678)
I20260809 23:14:27.197350 12970 tablet_copy_service.cc:161] P 60c9c82a25364f28818b186daee44f4c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ea2558e1baef47b2b24b355d6270d1e3 at {username='oryx', principal='oryx/127.2.76.253@KRBTEST.COM'} at 127.0.0.1:37678: session id = ea2558e1baef47b2b24b355d6270d1e3-00000000000000000000000000000000
I20260809 23:14:27.197942 12970 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 23:14:27.198438 13760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 23:14:27.199141 13760 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: tablet copy: Starting download of 0 data blocks...
I20260809 23:14:27.199299 13760 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: tablet copy: Starting download of 1 WAL segments...
I20260809 23:14:27.200217 13760 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 23:14:27.200806 12970 tablet_copy_service.cc:342] P 60c9c82a25364f28818b186daee44f4c: Request end of tablet copy session ea2558e1baef47b2b24b355d6270d1e3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.76.253@KRBTEST.COM'} at 127.0.0.1:37678
I20260809 23:14:27.200856 12970 tablet_copy_service.cc:433] P 60c9c82a25364f28818b186daee44f4c: ending tablet copy session ea2558e1baef47b2b24b355d6270d1e3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ea2558e1baef47b2b24b355d6270d1e3
I20260809 23:14:27.203112 13760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:27.203971 13959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:27.204043 13954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:27.204159 13955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:27.204640 13760 server_base.cc:1034] running on GCE node
I20260809 23:14:27.204728 13760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:27.204831 13760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:27.205958 13760 hybrid_clock.cc:648] HybridClock initialized: now 1786317267205950 us; error 21 us; skew 500 ppm
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:27.207064 13760 init.cc:377] Logged in from keytab as oryx/127.2.76.253@KRBTEST.COM (short username oryx)
I20260809 23:14:27.207288 13760 webserver.cc:460] Webserver started at http://127.2.76.253:33241/ using document root <none> and password file <none>
I20260809 23:14:27.207396 13760 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:27.207440 13760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:27.207929 13760 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.208434 13969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.208590 13760 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.208642 13760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/wal
uuid: "ea2558e1baef47b2b24b355d6270d1e3"
format_stamp: "Formatted at 2026-08-09 23:14:27 on dist-test-slave-7vjv"
server_key: "25fbf3a9a84f2e54c6ebb89cd10776dc"
server_key_iv: "f7b130dc349752c97ba6e6d60d57633c"
server_key_version: "encryptionkey@0"
I20260809 23:14:27.208679 13760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:27.218214 13760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:27.219614 13760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:27.222666 13760 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.253:40323
I20260809 23:14:27.222971 14064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.253:40323 every 8 connection(s)
I20260809 23:14:27.223121 13760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-1/data/info.pb
I20260809 23:14:27.223829 14065 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:14:27.224296 14065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: Bootstrap starting.
I20260809 23:14:27.226552 14065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:14:27.226795 14065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: Bootstrap complete.
I20260809 23:14:27.226974 14065 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: NON_VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: true } }
I20260809 23:14:27.227056 14065 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ea2558e1baef47b2b24b355d6270d1e3, State: Initialized, Role: LEARNER
I20260809 23:14:27.227106 14065 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: NON_VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: true } }
I20260809 23:14:27.227298 14065 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:27.227294 14071 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: NON_VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: true } } }
I20260809 23:14:27.227413 14071 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [sys.catalog]: This master's current role is: LEARNER
I20260809 23:14:27.228719 14095 catalog_manager.cc:1259] Loaded cluster ID: d4c48d90d186463aa8ea2ca835a219ea
I20260809 23:14:27.228756 14095 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: loading cluster ID for follower catalog manager: success
I20260809 23:14:27.229593 14095 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: acquiring CA information for follower catalog manager: success
I20260809 23:14:27.229830 14095 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:14:27.246734 14096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.240425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59282 (local address 127.2.76.253:40323)
0809 23:14:27.240536 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.240538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.240549 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:14:27.240573 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.240574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.240585 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.240682 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.241187 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.242066 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.242739 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.243015 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.243099 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.244176 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.244177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.244178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.244185 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.245695 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.245996 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.245997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.245997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.246063 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.246240 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.246241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.246241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.246369 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.246468 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.246540 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.246633 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
WARNING: no policy specified for oryx/127.2.76.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.76.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.76.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.252.keytab.
Entry for principal oryx/127.2.76.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.252.keytab.
I20260809 23:14:27.273022 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.252:40671
--webserver_interface=127.2.76.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.252.keytab
--principal=oryx/127.2.76.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:37751,127.2.76.253:40323,127.2.76.252:40671
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:27.351388 14099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:27.351594 14099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:27.351629 14099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:27.352749 14099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:27.352807 14099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:27.352833 14099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:27.352872 14099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:27.352898 14099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:14:27.354210 14099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:37751,127.2.76.253:40323,127.2.76.252:40671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.252.keytab
--principal=oryx/127.2.76.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.252:40671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.76.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:27.354458 14099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:27.354669 14099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:27.358841 14114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:27.358973 14106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:27.358943 14108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:27.359120 14099 server_base.cc:1034] running on GCE node
I20260809 23:14:27.359347 14099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:27.359635 14099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:27.360792 14099 hybrid_clock.cc:648] HybridClock initialized: now 1786317267360786 us; error 42 us; skew 500 ppm
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:27.362964 14099 init.cc:377] Logged in from keytab as oryx/127.2.76.252@KRBTEST.COM (short username oryx)
I20260809 23:14:27.363325 14099 webserver.cc:460] Webserver started at http://127.2.76.252:44463/ using document root <none> and password file <none>
I20260809 23:14:27.363529 14099 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:27.363590 14099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:27.363723 14099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:27.364694 14099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/data/instance:
uuid: "ce0985f9218f49629769fe7d73a8e43c"
format_stamp: "Formatted at 2026-08-09 23:14:27 on dist-test-slave-7vjv"
server_key: "5e39a2b2788c58d7eea14b3dbda7d4cf"
server_key_iv: "2be2f83b929d079fcb8413a49bcb387b"
server_key_version: "encryptionkey@0"
I20260809 23:14:27.365046 14099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/wal/instance:
uuid: "ce0985f9218f49629769fe7d73a8e43c"
format_stamp: "Formatted at 2026-08-09 23:14:27 on dist-test-slave-7vjv"
server_key: "5e39a2b2788c58d7eea14b3dbda7d4cf"
server_key_iv: "2be2f83b929d079fcb8413a49bcb387b"
server_key_version: "encryptionkey@0"
I20260809 23:14:27.366417 14099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:14:27.367090 14124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.367264 14099 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:27.367367 14099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/wal
uuid: "ce0985f9218f49629769fe7d73a8e43c"
format_stamp: "Formatted at 2026-08-09 23:14:27 on dist-test-slave-7vjv"
server_key: "5e39a2b2788c58d7eea14b3dbda7d4cf"
server_key_iv: "2be2f83b929d079fcb8413a49bcb387b"
server_key_version: "encryptionkey@0"
I20260809 23:14:27.367452 14099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:27.376144 14099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:27.377696 14099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:27.377859 14099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:27.383476 14099 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.252:40671
I20260809 23:14:27.383512 14215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.252:40671 every 8 connection(s)
I20260809 23:14:27.383919 14099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/data/info.pb
I20260809 23:14:27.384877 14216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:27.386314 14216 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:14:27.388875 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 14099
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.252@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.76.252@KRBTEST.COM
I20260809 23:14:27.395313 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.386992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37680 (local address 127.2.76.254:37751)
0809 23:14:27.387058 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.387060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.387172 (+ 112us) server_negotiation.cc:407] Connection header received
0809 23:14:27.387699 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.387701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.387713 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.387761 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.388385 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.388830 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.390161 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.390423 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.390491 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.392588 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.392589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.392589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.392598 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.393852 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.394298 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.394300 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.394300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.394365 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.394668 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.394668 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.394669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.394787 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.394856 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.395139 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.395225 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 23:14:27.395530 14221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.386826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:37680)
0809 23:14:27.387061 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.387104 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.387576 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.387824 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.387827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.388037 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.388297 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.388301 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.388920 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.388921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.390081 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.390083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.390505 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.390506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.390602 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.391018 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.391025 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.392494 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.393936 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.393942 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.393944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.394220 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.394439 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.394440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.394440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.394597 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.394863 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.394870 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.394982 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.395102 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.395291 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":137,"threads_started":1}
I20260809 23:14:27.396056 14216 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:14:27.397696 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.389460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46974 (local address 127.2.76.252:40671)
0809 23:14:27.389632 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.389636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.389657 (+ 21us) server_negotiation.cc:407] Connection header received
0809 23:14:27.389678 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.389679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.389697 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.389775 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.390318 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.391151 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.391972 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.392380 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.392447 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.394831 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.394838 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.394839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.394845 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.396671 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.396968 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.396971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.396971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.397024 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.397220 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.397220 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.397221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.397342 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.397492 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.397539 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.397606 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":100,"threads_started":1}
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.252@KRBTEST.COM for oryx/127.2.76.253@KRBTEST.COM
I20260809 23:14:27.401599 14221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.396285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:40323 (local address 127.0.0.1:59284)
0809 23:14:27.396330 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.396341 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.396375 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.396544 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.396545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.396709 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.396752 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.396753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.397419 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.397419 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.398053 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.398055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.398453 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.398454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.398501 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.398752 (+ 251us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.398755 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.399722 (+ 967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.400956 (+ 1234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.400957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.400957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.401122 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.401244 (+ 122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.401244 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.401244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.401281 (+ 37us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.401427 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.401427 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.401455 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.401496 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.401552 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 23:14:27.401618 14096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.396371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59284 (local address 127.2.76.253:40323)
0809 23:14:27.396418 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.396420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.396429 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:14:27.396453 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.396454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.396470 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.396540 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.396832 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.397335 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.398129 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.398392 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.398442 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.399792 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.399793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.399793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.399799 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.400897 (+ 1098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.401173 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.401174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.401175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.401212 (+ 37us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.401310 (+ 98us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.401310 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.401310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.401384 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.401422 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.401512 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.401565 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:14:27.401978 14216 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.252" port: 40671 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.252@KRBTEST.COM for oryx/127.2.76.252@KRBTEST.COM
I20260809 23:14:27.407065 14221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.402310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:40671 (local address 127.0.0.1:46984)
0809 23:14:27.402353 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.402362 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.402395 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.402480 (+ 85us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.402480 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.402576 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.402599 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.402600 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.403089 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.403090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.403477 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.403478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.403758 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.403758 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.403790 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.404148 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.404153 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.405137 (+ 984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.406402 (+ 1265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.406403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.406403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.406546 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.406703 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.406703 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.406703 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.406749 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.406884 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.406885 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.406921 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.406958 (+ 37us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.407021 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 23:14:27.407131 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.402353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46984 (local address 127.2.76.252:40671)
0809 23:14:27.402405 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.402406 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.402412 (+ 6us) server_negotiation.cc:407] Connection header received
0809 23:14:27.402429 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.402430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.402436 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.402478 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.402649 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.403029 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.403537 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.403701 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.403744 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.405204 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.405205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.405205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.405211 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.406329 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.406604 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.406605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.406605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.406651 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.406783 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.406783 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.406784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.406854 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.406902 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.406971 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.407060 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":36}
I20260809 23:14:27.408244 14216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: Bootstrap starting.
I20260809 23:14:27.409027 14216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:27.409420 14216 log.cc:824] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:27.410065 14216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: No bootstrap required, opened a new log
I20260809 23:14:27.411350 14216 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 40671 } }
I20260809 23:14:27.411470 14216 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:27.411514 14216 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0985f9218f49629769fe7d73a8e43c, State: Initialized, Role: FOLLOWER
I20260809 23:14:27.411628 14216 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 40671 } }
I20260809 23:14:27.411871 14216 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:27.411881 14227 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 40671 } } }
I20260809 23:14:27.411999 14227 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [sys.catalog]: This master's current role is: FOLLOWER
W20260809 23:14:27.413581 14247 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:14:27.413640 14247 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:14:27.415647 14099 master_runner.cc:428] Detected that this master ce0985f9218f49629769fe7d73a8e43c is joining an existing cluster
I20260809 23:14:27.415684 14099 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.76.252:40671
I20260809 23:14:27.422976 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.417278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46990 (local address 127.2.76.252:40671)
0809 23:14:27.417304 (+ 26us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.417305 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.417387 (+ 82us) server_negotiation.cc:407] Connection header received
0809 23:14:27.417461 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.417462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.417469 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.417521 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.417886 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.418506 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.419621 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.419846 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.419875 (+ 29us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.420711 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.420712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.420712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.420718 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.421881 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.422209 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.422210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.422210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.422256 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.422474 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.422474 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.422475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.422574 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.422652 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.422786 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.422897 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":12}
I20260809 23:14:27.422960 14263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.417153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:40671 (local address 127.0.0.1:46990)
0809 23:14:27.417343 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.417350 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.417403 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.417526 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.417526 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.417736 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.417811 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.417813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.418590 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.418590 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.419528 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.419531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.419935 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.419935 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.419978 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.420347 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.420352 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.420627 (+ 275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.421973 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.421974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.421974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.422143 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.422353 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.422354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.422354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.422402 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.422646 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.422646 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.422690 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.422767 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.422854 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":61,"threads_started":1}
I20260809 23:14:27.443773 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.416931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37690 (local address 127.2.76.254:37751)
0809 23:14:27.416968 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.416969 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.417158 (+ 189us) server_negotiation.cc:407] Connection header received
0809 23:14:27.417245 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.417246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.417255 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.417282 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.417861 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.418518 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.419797 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.420041 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.420080 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.441083 (+ 21003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.441084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.441084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.441090 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.442563 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.442997 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.442998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.442998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.443055 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.443254 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.443255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.443256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.443385 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.443451 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.443622 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.443690 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:14:27.443789 14261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.416926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:37690)
0809 23:14:27.417105 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.417113 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.417181 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.417338 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.417338 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.417690 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.417761 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.417763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.418596 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.418597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.419726 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.419728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.420178 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.420179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.420249 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.440636 (+ 20387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.440641 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.440972 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.442673 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.442674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.442675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.442921 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.443127 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.443127 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.443127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.443183 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.443454 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.443456 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.443496 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.443588 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.443690 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":41,"thread_start_us":87,"threads_started":1}
I20260809 23:14:27.443876 14096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.417004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59290 (local address 127.2.76.253:40323)
0809 23:14:27.417040 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.417041 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.417245 (+ 204us) server_negotiation.cc:407] Connection header received
0809 23:14:27.417324 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.417326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.417333 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.417358 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.417824 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.418512 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.419806 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.420063 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.420125 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.441209 (+ 21084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.441210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.441211 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.441217 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.442709 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.443097 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.443098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.443099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.443155 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.443332 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.443333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.443333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.443464 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.443537 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.443685 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.443767 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:14:27.444442 14262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.417058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:40323 (local address 127.0.0.1:59290)
0809 23:14:27.417189 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.417197 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.417266 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.417408 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.417409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.417661 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.417723 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.417724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.418608 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.418609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.419721 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.419723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.420192 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.420193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.420256 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.440726 (+ 20470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.440731 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.441120 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.442795 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.442796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.442796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.443023 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.443221 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.443222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.443222 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.443275 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.443530 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.443530 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.443571 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.444182 (+ 611us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.444275 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":78,"thread_start_us":77,"threads_started":1}
I20260809 23:14:27.448743 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 13040
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317266, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.254@KRBTEST.COM for oryx/127.2.76.252@KRBTEST.COM
I20260809 23:14:27.452855 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.445958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:40671 (local address 127.0.0.1:46994)
0809 23:14:27.445999 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.446009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.446049 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.446187 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.446188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.446303 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.446338 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.446339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.447197 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.447198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.447674 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.447675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.448179 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.448179 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.448234 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.448646 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.448651 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.450010 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.451724 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.451725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.451726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.451929 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.452204 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.452205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.452205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.452269 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.452523 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.452524 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.452565 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.452622 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.452722 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 23:14:27.453361 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.446004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46994 (local address 127.2.76.252:40671)
0809 23:14:27.446041 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.446043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.446051 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:27.446114 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.446115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.446124 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.446171 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.446419 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.447120 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.447753 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.448052 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.448180 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.450110 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.450111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.450111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.450117 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.451637 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.452011 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.452012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.452013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.452091 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.452331 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.452332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.452332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.452447 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.452498 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.453170 (+ 672us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.453257 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:14:27.457410 12928 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.76.252:40671: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c"
member_type: NON_VOTER
last_known_addr {
host: "127.2.76.252"
port: 40671
}
attrs {
promote: true
}
}
dest_uuid: "60c9c82a25364f28818b186daee44f4c"
cas_config_opid_index: 5
I20260809 23:14:27.457643 12928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: NON_VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: true } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } }
I20260809 23:14:27.458302 13933 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ce0985f9218f49629769fe7d73a8e43c (127.2.76.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: NON_VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: true } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } } }
I20260809 23:14:27.458721 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:40739
--local_ip_for_outbound_sockets=127.2.76.193
--tserver_master_addrs=127.2.76.254:37751
--webserver_port=33125
--webserver_interface=127.2.76.193
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.193.keytab
--principal=oryx/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.76.254:37751,127.2.76.253:40323
--tserver_master_addrs=127.2.76.254:37751,127.2.76.253:40323,127.2.76.252:40671 with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:14:27.459249 13933 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.76.252:40671
I20260809 23:14:27.459311 13933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "60c9c82a25364f28818b186daee44f4c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: NON_VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: true } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } } }
I20260809 23:14:27.459378 13933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:27.459777 14166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:27.460294 14099 master.cc:537] Master@127.2.76.252:40671 shutting down...
I20260809 23:14:27.461272 14166 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [term 1 FOLLOWER]: Refusing update from remote peer 60c9c82a25364f28818b186daee44f4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 23:14:27.461968 13934 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 23:14:27.462725 12906 consensus_peers.cc:487] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c -> Peer ce0985f9218f49629769fe7d73a8e43c (127.2.76.252:40671): Couldn't send request to peer ce0985f9218f49629769fe7d73a8e43c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 23:14:27.464377 14099 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:27.464468 14099 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:27.464514 14099 tablet_replica.cc:330] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: stopping tablet replica
I20260809 23:14:27.465853 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.458213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:40323 (local address 127.0.0.1:59292)
0809 23:14:27.459649 (+ 1436us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.459659 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.459697 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.459894 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.459895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.459896 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:27.459926 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.459927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.463683 (+ 3756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.463685 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.464833 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.464836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.465565 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.465567 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.465640 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.465664 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.465727 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.465768 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1422}
I20260809 23:14:27.466193 14096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.458297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59292 (local address 127.2.76.253:40323)
0809 23:14:27.458335 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.458336 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.459744 (+ 1408us) server_negotiation.cc:407] Connection header received
0809 23:14:27.459765 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.459766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.459788 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.459837 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:27.460010 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.460685 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.464940 (+ 4255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.465476 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.465546 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.465615 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.466070 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.466113 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:14:27.466655 14020 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 LEARNER]: Refusing update from remote peer 60c9c82a25364f28818b186daee44f4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 23:14:27.467002 13934 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: NON_VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 23:14:27.467628 14020 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ce0985f9218f49629769fe7d73a8e43c (127.2.76.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: NON_VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: true } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } } }
I20260809 23:14:27.469143 14071 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60c9c82a25364f28818b186daee44f4c. Latest consensus state: current_term: 1 leader_uuid: "60c9c82a25364f28818b186daee44f4c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: NON_VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: true } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } } }
I20260809 23:14:27.469238 14071 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [sys.catalog]: This master's current role is: LEARNER
I20260809 23:14:27.469280 14071 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "60c9c82a25364f28818b186daee44f4c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: NON_VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: true } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } } }
I20260809 23:14:27.469332 14071 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [sys.catalog]: This master's current role is: LEARNER
I20260809 23:14:27.469568 13934 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c: attempting to promote NON_VOTER ea2558e1baef47b2b24b355d6270d1e3 to VOTER
I20260809 23:14:27.469749 13934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } }
I20260809 23:14:27.470283 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.470177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:40671 (local address 127.0.0.1:47006)
0809 23:14:27.470224 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.470259 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:40671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 23:14:27.470386 14020 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 LEARNER]: Refusing update from remote peer 60c9c82a25364f28818b186daee44f4c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 23:14:27.470567 13933 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 23:14:27.470841 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.470450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:40671 (local address 127.0.0.1:47020)
0809 23:14:27.470540 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.470559 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:40671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":61,"threads_started":1}
W20260809 23:14:27.470925 12906 consensus_peers.cc:487] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c -> Peer ce0985f9218f49629769fe7d73a8e43c (127.2.76.252:40671): Couldn't send request to peer ce0985f9218f49629769fe7d73a8e43c. Status: Network error: Client connection negotiation failed: client connection to 127.2.76.252:40671: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 23:14:27.471336 13933 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ea2558e1baef47b2b24b355d6270d1e3 (127.2.76.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } } }
I20260809 23:14:27.472141 13934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "60c9c82a25364f28818b186daee44f4c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } } }
I20260809 23:14:27.472200 13934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:27.472055 13933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "60c9c82a25364f28818b186daee44f4c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } } }
I20260809 23:14:27.472347 13933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:27.472443 14020 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ea2558e1baef47b2b24b355d6270d1e3 (127.2.76.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } } }
I20260809 23:14:27.473358 14268 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "60c9c82a25364f28818b186daee44f4c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } } }
I20260809 23:14:27.473462 14268 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:14:27.478248 14099 master.cc:552] Master@127.2.76.252:40671 shutdown complete.
I20260809 23:14:27.478297 14099 master_runner.cc:305] Clearing existing system tablet
I20260809 23:14:27.478422 14099 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:27.478469 14099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:27.479202 14099 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:14:27.479761 14272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.479890 14099 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:27.479934 14099 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:14:27.479956 14099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/wal
uuid: "ce0985f9218f49629769fe7d73a8e43c"
format_stamp: "Formatted at 2026-08-09 23:14:27 on dist-test-slave-7vjv"
server_key: "5e39a2b2788c58d7eea14b3dbda7d4cf"
server_key_iv: "2be2f83b929d079fcb8413a49bcb387b"
server_key_version: "encryptionkey@0"
I20260809 23:14:27.480720 14099 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:27.482071 14099 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 23:14:27.482143 14099 log.cc:1196] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 23:14:27.482391 14099 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: Deleting consensus metadata
I20260809 23:14:27.482515 14099 master_runner.cc:315] Copying system tablet from 127.2.76.254:37751
I20260809 23:14:27.483071 14099 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: tablet copy: Beginning tablet copy session from remote peer at address 127.2.76.254:37751
I20260809 23:14:27.489944 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.483572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37702 (local address 127.2.76.254:37751)
0809 23:14:27.483607 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.483609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.483616 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:14:27.483635 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.483635 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.483644 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.483701 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.484214 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.484902 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.485813 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.486080 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.486145 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.487220 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.487221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.487222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.487229 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.488820 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.489198 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.489199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.489200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.489268 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.489482 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.489483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.489483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.489598 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.489654 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.489801 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.489870 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:14:27.489959 14281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.483309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:37702)
0809 23:14:27.483438 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.483449 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.483502 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.483744 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.483745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.483952 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.484003 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.484005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.485007 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.485008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.485723 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.485725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.486193 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.486194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.486252 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.486727 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.486732 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.487124 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.488909 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.488910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.488911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.489122 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.489356 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.489356 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.489357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.489420 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.489659 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.489660 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.489704 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.489766 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.489858 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
I20260809 23:14:27.490391 12970 tablet_copy_service.cc:140] P 60c9c82a25364f28818b186daee44f4c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ce0985f9218f49629769fe7d73a8e43c ({username='oryx', principal='oryx/127.2.76.252@KRBTEST.COM'} at 127.0.0.1:37702)
I20260809 23:14:27.490435 12970 tablet_copy_service.cc:161] P 60c9c82a25364f28818b186daee44f4c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ce0985f9218f49629769fe7d73a8e43c at {username='oryx', principal='oryx/127.2.76.252@KRBTEST.COM'} at 127.0.0.1:37702: session id = ce0985f9218f49629769fe7d73a8e43c-00000000000000000000000000000000
I20260809 23:14:27.490904 12970 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 23:14:27.491326 14099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 23:14:27.492758 14099 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: tablet copy: Starting download of 0 data blocks...
I20260809 23:14:27.493048 14099 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: tablet copy: Starting download of 1 WAL segments...
I20260809 23:14:27.493911 14099 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 23:14:27.494861 12970 tablet_copy_service.cc:342] P 60c9c82a25364f28818b186daee44f4c: Request end of tablet copy session ce0985f9218f49629769fe7d73a8e43c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.76.252@KRBTEST.COM'} at 127.0.0.1:37702
I20260809 23:14:27.494923 12970 tablet_copy_service.cc:433] P 60c9c82a25364f28818b186daee44f4c: ending tablet copy session ce0985f9218f49629769fe7d73a8e43c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ce0985f9218f49629769fe7d73a8e43c
I20260809 23:14:27.497802 14099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:27.498955 14282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:27.500856 14283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:27.501008 14286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:27.504315 14099 server_base.cc:1034] running on GCE node
I20260809 23:14:27.504413 14099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:27.504532 14099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:27.505658 14099 hybrid_clock.cc:648] HybridClock initialized: now 1786317267505637 us; error 38 us; skew 500 ppm
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:27.507081 14099 init.cc:377] Logged in from keytab as oryx/127.2.76.252@KRBTEST.COM (short username oryx)
I20260809 23:14:27.507347 14099 webserver.cc:460] Webserver started at http://127.2.76.252:44621/ using document root <none> and password file <none>
I20260809 23:14:27.507465 14099 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:27.507503 14099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:27.508286 14099 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:14:27.508857 14298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.508972 14099 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.509013 14099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/wal
uuid: "ce0985f9218f49629769fe7d73a8e43c"
format_stamp: "Formatted at 2026-08-09 23:14:27 on dist-test-slave-7vjv"
server_key: "5e39a2b2788c58d7eea14b3dbda7d4cf"
server_key_iv: "2be2f83b929d079fcb8413a49bcb387b"
server_key_version: "encryptionkey@0"
I20260809 23:14:27.509068 14099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:27.524700 14099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:27.526500 14099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:27.529565 14099 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.252:40671
I20260809 23:14:27.529654 14388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.252:40671 every 8 connection(s)
I20260809 23:14:27.529866 14099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/master-2/data/info.pb
I20260809 23:14:27.530651 14390 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:14:27.531193 14390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: Bootstrap starting.
I20260809 23:14:27.534413 14390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:14:27.534754 14390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: Bootstrap complete.
I20260809 23:14:27.535045 14390 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } }
I20260809 23:14:27.535233 14390 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ce0985f9218f49629769fe7d73a8e43c, State: Initialized, Role: LEARNER
I20260809 23:14:27.535343 14390 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } }
I20260809 23:14:27.535643 14390 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:27.535667 14395 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } } }
I20260809 23:14:27.535781 14395 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c [sys.catalog]: This master's current role is: LEARNER
I20260809 23:14:27.537520 14412 catalog_manager.cc:1259] Loaded cluster ID: d4c48d90d186463aa8ea2ca835a219ea
I20260809 23:14:27.537561 14412 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: loading cluster ID for follower catalog manager: success
I20260809 23:14:27.538779 14412 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: acquiring CA information for follower catalog manager: success
I20260809 23:14:27.539050 14412 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ce0985f9218f49629769fe7d73a8e43c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 23:14:27.553324 14264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:27.553529 14264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:27.553578 14264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:27.554695 14264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:27.554755 14264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:27.554804 14264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:14:27.556342 14264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.193.keytab
--principal=oryx/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:40739
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=33125
--tserver_master_addrs=127.2.76.254:37751,127.2.76.253:40323,127.2.76.252:40671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:27.556582 14264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:27.556958 14264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:27.562333 14418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:27.562364 14425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:27.562368 14420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:27.562860 14264 server_base.cc:1034] running on GCE node
I20260809 23:14:27.563054 14264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:27.563278 14264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:27.564427 14264 hybrid_clock.cc:648] HybridClock initialized: now 1786317267564414 us; error 29 us; skew 500 ppm
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:27.567129 14264 init.cc:377] Logged in from keytab as oryx/127.2.76.193@KRBTEST.COM (short username oryx)
I20260809 23:14:27.567523 14264 webserver.cc:460] Webserver started at http://127.2.76.193:33125/ using document root <none> and password file <none>
I20260809 23:14:27.567718 14264 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:27.567788 14264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:27.569036 14264 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:27.569545 14435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.569694 14264 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:27.569785 14264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "66402c0016014fbbbc0e6c094ea5b551"
format_stamp: "Formatted at 2026-08-09 23:14:26 on dist-test-slave-7vjv"
server_key: "9a0b6a68cf25454703d5afed7cb1fca9"
server_key_iv: "7dbe443cb6f167db53505e5e49dbf299"
server_key_version: "encryptionkey@0"
I20260809 23:14:27.570056 14264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:27.578058 14264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:27.580056 14264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:27.580240 14264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:27.580447 14264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:27.580713 14264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:27.580762 14264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.580798 14264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:27.580823 14264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.588620 14264 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:40739
I20260809 23:14:27.588652 14625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:40739 every 8 connection(s)
I20260809 23:14:27.588976 14264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.253@KRBTEST.COM
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.252@KRBTEST.COM
I20260809 23:14:27.596590 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 14264
I20260809 23:14:27.596769 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 13278
I20260809 23:14:27.606988 14635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.589629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:40323 (local address 127.2.76.193:38797)
0809 23:14:27.590186 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.590195 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.590935 (+ 740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.591167 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.591170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.591474 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.591627 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.591629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.592345 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.592346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.593199 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.593201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.593628 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.593630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.593693 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.594093 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.594097 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.595629 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.605706 (+ 10077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.605712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.605714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.605943 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.606101 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.606101 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.606102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.606197 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.606439 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.606444 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.606565 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.606651 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.606785 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":11,"thread_start_us":74,"threads_started":1}
I20260809 23:14:27.607354 14096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.589910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:38797 (local address 127.2.76.253:40323)
0809 23:14:27.589955 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.589957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.590238 (+ 281us) server_negotiation.cc:407] Connection header received
0809 23:14:27.591064 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.591065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.591074 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.591116 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.591739 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.592253 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.593286 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.593558 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.593617 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.595721 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.595722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.595722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.595730 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.604512 (+ 8782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.606006 (+ 1494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.606006 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.606007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.606053 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.606244 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.606245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.606245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.606373 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.606419 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.607167 (+ 748us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.607236 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:14:27.608718 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.589979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:48419 (local address 127.2.76.254:37751)
0809 23:14:27.590025 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.590027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.590340 (+ 313us) server_negotiation.cc:407] Connection header received
0809 23:14:27.591023 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.591024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.591036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.591065 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.591793 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.592431 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.593191 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.593407 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.593464 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.596205 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.596206 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.596207 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.596214 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.605634 (+ 9420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.608050 (+ 2416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.608051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.608051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.608118 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.608262 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.608263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.608263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.608374 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.608406 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.608547 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.608624 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:14:27.609028 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.589885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:48419)
0809 23:14:27.590277 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.590283 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.590935 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.591167 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.591170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.591477 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.591720 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.591722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.592509 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.592510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.593109 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.593111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.593480 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.593481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.593589 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.593970 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.593977 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.596060 (+ 2083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.607807 (+ 11747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.607808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.607808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.607980 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.608172 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.608173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.608173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.608216 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.608436 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.608437 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.608468 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.608873 (+ 405us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.608966 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":204,"threads_started":1}
I20260809 23:14:27.609367 14627 heartbeater.cc:347] Connected to a master server at 127.2.76.254:37751
I20260809 23:14:27.609449 14627 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:27.609599 14627 heartbeater.cc:511] Master 127.2.76.254:37751 requested a full tablet report, sending...
I20260809 23:14:27.610001 12928 ts_manager.cc:194] Re-registered known tserver with Master: 66402c0016014fbbbc0e6c094ea5b551 (127.2.76.193:40739)
I20260809 23:14:27.610333 12928 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:48419
I20260809 23:14:27.616688 14628 heartbeater.cc:347] Connected to a master server at 127.2.76.253:40323
I20260809 23:14:27.616744 14628 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:27.616801 14628 heartbeater.cc:511] Master 127.2.76.253:40323 requested a full tablet report, sending...
I20260809 23:14:27.617118 13996 ts_manager.cc:194] Registered new tserver with Master: 66402c0016014fbbbc0e6c094ea5b551 (127.2.76.193:40739)
I20260809 23:14:27.620297 14633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.589628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:40671 (local address 127.2.76.193:55053)
0809 23:14:27.590070 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.590109 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.590918 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.591167 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.591170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.591468 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.591618 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.591621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.592652 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.592653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.593330 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.593331 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.593700 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.593701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.593774 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.594280 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.594285 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.596705 (+ 2420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.619176 (+ 22471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.619177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.619177 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.619328 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.619523 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.619524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.619524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.619587 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.619885 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.619886 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.619929 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.619992 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.620064 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":87,"threads_started":1}
I20260809 23:14:27.620299 14631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.589621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:55053 (local address 127.2.76.252:40671)
0809 23:14:27.589718 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.589720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.590172 (+ 452us) server_negotiation.cc:407] Connection header received
0809 23:14:27.591011 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.591012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.591026 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.591092 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.591702 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.592580 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.593396 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.593638 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.593694 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.596811 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.596812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.596812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.596820 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.619083 (+ 22263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.619401 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.619402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.619402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.619458 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.619661 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.619662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.619662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.619797 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.619858 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.620037 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.620170 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":48,"threads_started":1}
I20260809 23:14:27.620671 14626 heartbeater.cc:347] Connected to a master server at 127.2.76.252:40671
I20260809 23:14:27.620718 14626 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:27.620765 14626 heartbeater.cc:511] Master 127.2.76.252:40671 requested a full tablet report, sending...
I20260809 23:14:27.621057 14314 ts_manager.cc:194] Registered new tserver with Master: 66402c0016014fbbbc0e6c094ea5b551 (127.2.76.193:40739)
I20260809 23:14:27.622044 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:34053
--local_ip_for_outbound_sockets=127.2.76.194
--tserver_master_addrs=127.2.76.254:37751
--webserver_port=34447
--webserver_interface=127.2.76.194
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.194.keytab
--principal=oryx/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.76.254:37751,127.2.76.253:40323
--tserver_master_addrs=127.2.76.254:37751,127.2.76.253:40323,127.2.76.252:40671 with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:27.695612 14637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:27.695827 14637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:27.695853 14637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:27.696923 14637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:27.696965 14637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:27.697012 14637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:14:27.698268 14637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.194.keytab
--principal=oryx/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:34053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=34447
--tserver_master_addrs=127.2.76.254:37751,127.2.76.253:40323,127.2.76.252:40671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:27.698483 14637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:27.698700 14637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:27.703297 14647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:27.703303 14650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:27.703459 14646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:27.703512 14637 server_base.cc:1034] running on GCE node
I20260809 23:14:27.703708 14637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:27.703922 14637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:27.705075 14637 hybrid_clock.cc:648] HybridClock initialized: now 1786317267705061 us; error 23 us; skew 500 ppm
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:27.707300 14637 init.cc:377] Logged in from keytab as oryx/127.2.76.194@KRBTEST.COM (short username oryx)
I20260809 23:14:27.707751 14637 webserver.cc:460] Webserver started at http://127.2.76.194:34447/ using document root <none> and password file <none>
I20260809 23:14:27.707911 14637 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:27.707957 14637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:27.709323 14637 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:27.709767 14662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.709972 14637 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.710081 14637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "3d67646499fc4284afd0d289a2c9c04f"
format_stamp: "Formatted at 2026-08-09 23:14:26 on dist-test-slave-7vjv"
server_key: "f7c3366bc3575b5785778a5bcbd667aa"
server_key_iv: "5cb934708b4cc014dd0a5619f440a119"
server_key_version: "encryptionkey@0"
I20260809 23:14:27.710340 14637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:27.718978 14637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:27.720774 14637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:27.720904 14637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:27.721120 14637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:27.721374 14637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:27.721402 14637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.721419 14637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:27.721459 14637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.729166 14637 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:34053
I20260809 23:14:27.729223 14856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:34053 every 8 connection(s)
I20260809 23:14:27.729485 14637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.252@KRBTEST.COM
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.253@KRBTEST.COM
I20260809 23:14:27.736722 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 14637
I20260809 23:14:27.736889 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 13524
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
I20260809 23:14:27.740419 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.730614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:43091 (local address 127.2.76.254:37751)
0809 23:14:27.730656 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.730657 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.731643 (+ 986us) server_negotiation.cc:407] Connection header received
0809 23:14:27.731744 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.731745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.731755 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.731798 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.732362 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.732796 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.733679 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.733855 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.733923 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.737414 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.737415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.737415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.737423 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.738950 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.739401 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.739402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.739403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.739458 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.739731 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.739732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.739732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.739858 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.739897 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.740248 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.740324 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:14:27.740621 14096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.730432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:46781 (local address 127.2.76.253:40323)
0809 23:14:27.730475 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.730477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.730485 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:27.731549 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.731551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.731562 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.731610 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.732377 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.733032 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.734075 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.734326 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.734367 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.736879 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.736880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.736880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.736888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.738627 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.739798 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.739799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.739800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.739852 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.740034 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.740035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.740035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.740182 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.740222 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.740393 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.740487 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:14:27.740907 14863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.730419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:43091)
0809 23:14:27.731590 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.731600 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.731674 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.731800 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.731801 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.732055 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.732276 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.732280 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.732895 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.732895 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.733594 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.733596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.733931 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.733933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.734073 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.734668 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.734674 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.737322 (+ 2648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.739037 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.739044 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.739046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.739316 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.739524 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.739525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.739525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.739656 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.739939 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.739945 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.740108 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.740519 (+ 411us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.740692 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1154,"thread_start_us":53,"threads_started":1}
I20260809 23:14:27.740991 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.730170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:40323 (local address 127.2.76.194:46781)
0809 23:14:27.730333 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.730361 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.731419 (+ 1058us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.731653 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.731657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.732075 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.732277 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.732283 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.733122 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.733123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.733979 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.733982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.734443 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.734444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.734516 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.735012 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.735017 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.736781 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.739500 (+ 2719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.739501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.739501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.739717 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.739915 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.739915 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.739916 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.739967 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.740263 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.740264 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.740300 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.740718 (+ 418us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.740829 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":61,"thread_start_us":90,"threads_started":1}
I20260809 23:14:27.741123 14631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.730575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:42351 (local address 127.2.76.252:40671)
0809 23:14:27.730615 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.730617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.730625 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:27.731518 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.731519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.731531 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.731592 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.732377 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.733038 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.734064 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.734344 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.734430 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.736429 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.736430 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.736431 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.736438 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.739783 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.740135 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.740137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.740137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.740192 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.740374 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.740374 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.740375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.740518 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.740563 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.740952 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.741037 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:14:27.741658 14859 heartbeater.cc:347] Connected to a master server at 127.2.76.254:37751
I20260809 23:14:27.741761 14859 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:27.741837 14866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.730170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:40671 (local address 127.2.76.194:42351)
0809 23:14:27.730514 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.730524 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.731432 (+ 908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.731653 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.731657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.732081 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.732277 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.732282 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.733122 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.733123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.733979 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.733982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.734439 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.734440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.734507 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.734847 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.734851 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.736328 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.739854 (+ 3526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.739854 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.739855 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.740028 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.740260 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.740260 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.740260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.740313 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.740809 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.740809 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.740844 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.741553 (+ 709us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.741658 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":35,"thread_start_us":86,"threads_started":1}
I20260809 23:14:27.741935 14859 heartbeater.cc:511] Master 127.2.76.254:37751 requested a full tablet report, sending...
I20260809 23:14:27.742236 14857 heartbeater.cc:347] Connected to a master server at 127.2.76.252:40671
I20260809 23:14:27.742246 12928 ts_manager.cc:194] Re-registered known tserver with Master: 3d67646499fc4284afd0d289a2c9c04f (127.2.76.194:34053)
I20260809 23:14:27.742280 14857 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:27.742337 14857 heartbeater.cc:511] Master 127.2.76.252:40671 requested a full tablet report, sending...
I20260809 23:14:27.742570 14314 ts_manager.cc:194] Registered new tserver with Master: 3d67646499fc4284afd0d289a2c9c04f (127.2.76.194:34053)
I20260809 23:14:27.742678 12928 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:43091
I20260809 23:14:27.744273 14861 heartbeater.cc:347] Connected to a master server at 127.2.76.253:40323
I20260809 23:14:27.744326 14861 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:27.744388 14861 heartbeater.cc:511] Master 127.2.76.253:40323 requested a full tablet report, sending...
I20260809 23:14:27.745235 13996 ts_manager.cc:194] Registered new tserver with Master: 3d67646499fc4284afd0d289a2c9c04f (127.2.76.194:34053)
I20260809 23:14:27.746255 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:33227
--local_ip_for_outbound_sockets=127.2.76.195
--tserver_master_addrs=127.2.76.254:37751
--webserver_port=43159
--webserver_interface=127.2.76.195
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.195.keytab
--principal=oryx/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.76.254:37751,127.2.76.253:40323
--tserver_master_addrs=127.2.76.254:37751,127.2.76.253:40323,127.2.76.252:40671 with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:27.820888 14868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:27.821059 14868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:27.821081 14868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:27.822002 14868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:27.822041 14868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:27.822064 14868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:14:27.823283 14868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.195.keytab
--principal=oryx/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:33227
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=43159
--tserver_master_addrs=127.2.76.254:37751,127.2.76.253:40323,127.2.76.252:40671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:27.823452 14868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:27.823606 14868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:27.828313 14881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:27.828403 14883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:27.828536 14868 server_base.cc:1034] running on GCE node
W20260809 23:14:27.828411 14891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:27.828765 14868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:27.828969 14868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:27.830106 14868 hybrid_clock.cc:648] HybridClock initialized: now 1786317267830092 us; error 35 us; skew 500 ppm
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:27.832418 14868 init.cc:377] Logged in from keytab as oryx/127.2.76.195@KRBTEST.COM (short username oryx)
I20260809 23:14:27.832779 14868 webserver.cc:460] Webserver started at http://127.2.76.195:43159/ using document root <none> and password file <none>
I20260809 23:14:27.832965 14868 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:27.833035 14868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:27.834312 14868 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:27.834846 14902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.835014 14868 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.835115 14868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "fbd85ed7dd8d47058f5e51279d68ed39"
format_stamp: "Formatted at 2026-08-09 23:14:26 on dist-test-slave-7vjv"
server_key: "ea15635f084f623cde1a10279f0f82c0"
server_key_iv: "d16448d655673f52ba461dccc5b14f41"
server_key_version: "encryptionkey@0"
I20260809 23:14:27.835364 14868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:27.847241 14868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:27.848919 14868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:27.849016 14868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:27.849159 14868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:14:27.849361 14868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:27.849388 14868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.849406 14868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:27.849417 14868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:27.854899 14868 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:33227
I20260809 23:14:27.854970 15086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:33227 every 8 connection(s)
I20260809 23:14:27.855314 14868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:14:27.861096 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 14868
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.252@KRBTEST.COM
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
Aug 09 23:14:27 dist-test-slave-7vjv krb5kdc[12852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317267, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.253@KRBTEST.COM
I20260809 23:14:27.866201 15101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.855917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:40323 (local address 127.2.76.195:36287)
0809 23:14:27.856168 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.856177 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.856655 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.856902 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.856905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.857336 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.857565 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.857571 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.858487 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.858488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.859340 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.859342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.859814 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.859815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.859879 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.860491 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.860496 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.862882 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.864654 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.864661 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.864663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.864947 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.865137 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.865138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.865139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.865264 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.865570 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.865577 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.865707 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.865801 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.865964 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":45,"thread_start_us":89,"threads_started":1}
I20260809 23:14:27.866549 14096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.855961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:36287 (local address 127.2.76.253:40323)
0809 23:14:27.856016 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.856019 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.856230 (+ 211us) server_negotiation.cc:407] Connection header received
0809 23:14:27.856760 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.856761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.856774 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.856811 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.857676 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.858397 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.859436 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.859698 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.859760 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.862943 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.862944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.862944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.862951 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.864568 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.865019 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.865020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.865021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.865071 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.865340 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.865341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.865341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.865468 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.865542 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.866342 (+ 800us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.866425 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:14:27.866995 15092 heartbeater.cc:347] Connected to a master server at 127.2.76.253:40323
I20260809 23:14:27.867096 15092 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:27.867282 15092 heartbeater.cc:511] Master 127.2.76.253:40323 requested a full tablet report, sending...
I20260809 23:14:27.867681 13996 ts_manager.cc:194] Registered new tserver with Master: fbd85ed7dd8d47058f5e51279d68ed39 (127.2.76.195:33227)
I20260809 23:14:27.868073 15106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.856196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:40671 (local address 127.2.76.195:43309)
0809 23:14:27.856464 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.856472 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.856653 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.857181 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.857182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.857361 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.857563 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.857568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.858248 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.858249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.858977 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.858979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.859324 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.859325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.859442 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.860085 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.860135 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.863171 (+ 3036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.864960 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.864961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.864961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.865167 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.866436 (+ 1269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.866436 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.866437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.866494 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.867626 (+ 1132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.867627 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.867671 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.867746 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.867843 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":22,"thread_start_us":225,"threads_started":1}
I20260809 23:14:27.868528 14631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.856364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:43309 (local address 127.2.76.252:40671)
0809 23:14:27.856404 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.856405 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.856540 (+ 135us) server_negotiation.cc:407] Connection header received
0809 23:14:27.856825 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.856826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.856837 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.856880 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.857677 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.858152 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.859074 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.859251 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.859318 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.863282 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.863283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.863283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.863290 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.864878 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.866158 (+ 1280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.866159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.866160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.866227 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.867400 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.867401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.867402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.867533 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.867597 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.868360 (+ 763us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.868445 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:14:27.868994 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.855917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:36461)
0809 23:14:27.856110 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.856144 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.856638 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.856902 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.856905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.857336 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:27.857565 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.857571 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.858487 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.858488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.859340 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.859342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.859780 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.859781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.859847 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.860404 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:27.860410 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:27.862595 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:27.867079 (+ 4484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.867080 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.867081 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.867321 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.868310 (+ 989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:27.868311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:27.868312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:27.868372 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:27.868667 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:27.868668 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:27.868708 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.868765 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.868856 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":54,"thread_start_us":86,"threads_started":1}
I20260809 23:14:27.869264 15088 heartbeater.cc:347] Connected to a master server at 127.2.76.252:40671
I20260809 23:14:27.869313 15088 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:27.869321 12998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.856151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:36461 (local address 127.2.76.254:37751)
0809 23:14:27.856198 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.856199 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.856208 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:14:27.856763 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.856764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.856787 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.856833 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.857676 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.858394 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.859411 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.859705 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.859781 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.862682 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.862683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.862684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.862689 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.866968 (+ 4279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.868178 (+ 1210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.868179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.868179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.868235 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.868442 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.868443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.868444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.868580 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.868641 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.869142 (+ 501us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.869224 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:14:27.869374 15088 heartbeater.cc:511] Master 127.2.76.252:40671 requested a full tablet report, sending...
I20260809 23:14:27.869637 14314 ts_manager.cc:194] Registered new tserver with Master: fbd85ed7dd8d47058f5e51279d68ed39 (127.2.76.195:33227)
I20260809 23:14:27.869788 15091 heartbeater.cc:347] Connected to a master server at 127.2.76.254:37751
I20260809 23:14:27.869834 15091 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:27.869931 15091 heartbeater.cc:511] Master 127.2.76.254:37751 requested a full tablet report, sending...
I20260809 23:14:27.870149 12928 ts_manager.cc:194] Re-registered known tserver with Master: fbd85ed7dd8d47058f5e51279d68ed39 (127.2.76.195:33227)
I20260809 23:14:27.870579 12928 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:36461
I20260809 23:14:27.871604 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.862373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59294 (local address 127.2.76.253:40323)
0809 23:14:27.862797 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.862799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.862805 (+ 6us) server_negotiation.cc:407] Connection header received
0809 23:14:27.862823 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.862824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.862834 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.862884 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.863691 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.864376 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.865289 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.865570 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.865619 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.868440 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.868441 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.868441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.868448 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.870431 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.870795 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.870797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.870797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.870856 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.871060 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.871061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.871061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.871161 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.871256 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.871373 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.871452 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":55,"threads_started":1}
I20260809 23:14:27.873800 15112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.862095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37718 (local address 127.2.76.254:37751)
0809 23:14:27.863572 (+ 1477us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.863574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.863581 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:14:27.863602 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.863602 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.863613 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.864297 (+ 684us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.864332 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.865019 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.865934 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.866272 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.866582 (+ 310us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.868356 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.868357 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.868357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.868364 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.872671 (+ 4307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.873037 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.873039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.873039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.873094 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.873290 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.873291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.873291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.873396 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.873499 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.873622 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.873690 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1447,"thread_start_us":114,"threads_started":1}
I20260809 23:14:27.875305 15120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.862528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47026 (local address 127.2.76.252:40671)
0809 23:14:27.866064 (+ 3536us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.866066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.866071 (+ 5us) server_negotiation.cc:407] Connection header received
0809 23:14:27.866087 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.866088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.866116 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.866168 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.869220 (+ 3052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.869897 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.870799 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.871092 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.871149 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.872482 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:27.872483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:27.872483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:27.872489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:27.873864 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.874420 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.874422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.874422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.874487 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:27.874709 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:27.874711 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:27.874711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:27.874852 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:27.874947 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.875086 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.875180 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3520,"thread_start_us":50,"threads_started":1}
I20260809 23:14:27.875530 12928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:14:27.877092 12928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:14:27.883885 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.880498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:33227 (local address 127.0.0.1:51942)
0809 23:14:27.880565 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.880576 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.880614 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.881227 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.881228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.881230 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:27.881264 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.881266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.882033 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.882034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.882933 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.882935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.883649 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.883650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.883702 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.883706 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.883777 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.883823 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":23}
I20260809 23:14:27.884230 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.880688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51942 (local address 127.2.76.195:33227)
0809 23:14:27.881095 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.881099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.881107 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:27.881137 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.881138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.881151 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.881228 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:27.881311 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.881940 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.883023 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.883570 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.883649 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.883711 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.884062 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.884148 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":88,"threads_started":1}
I20260809 23:14:27.884240 15125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.880545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:34053 (local address 127.0.0.1:48838)
0809 23:14:27.880706 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.880715 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.880751 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.881365 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.881365 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.881367 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:27.881396 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.881397 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.882439 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.882440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.883191 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.883194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.883986 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.883987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.884050 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.884055 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.884136 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.884168 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":74,"threads_started":1}
I20260809 23:14:27.884483 15129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.880854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48838 (local address 127.2.76.194:34053)
0809 23:14:27.881052 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.881055 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.881064 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:14:27.881101 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.881104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.881119 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.881374 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:27.881444 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.882363 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.883251 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.883896 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.883976 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.884326 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.884359 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.884407 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":87,"threads_started":1}
I20260809 23:14:27.884749 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.880498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:40739 (local address 127.0.0.1:35008)
0809 23:14:27.880566 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.880575 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.880621 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.880988 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.880988 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.880990 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:27.881030 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.881031 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.882299 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.882300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.883270 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.883272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.884562 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.884563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.884620 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.884624 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.884655 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.884694 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 23:14:27.884809 14979 tablet_service.cc:1467] Processing CreateTablet for tablet 16bcfad782e545febf5ab8a8e05488f4 (DEFAULT_TABLE table=test-table [id=06b98188a49948fa924827666bfd21e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:27.885124 14979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16bcfad782e545febf5ab8a8e05488f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:27.885254 15126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.880643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35008 (local address 127.2.76.193:40739)
0809 23:14:27.880816 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.880820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.880831 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:14:27.880870 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.880872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.880888 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.880973 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:27.881116 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.882220 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.883333 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.884194 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.884896 (+ 702us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.885076 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.885103 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.885147 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260809 23:14:27.885130 14755 tablet_service.cc:1467] Processing CreateTablet for tablet 16bcfad782e545febf5ab8a8e05488f4 (DEFAULT_TABLE table=test-table [id=06b98188a49948fa924827666bfd21e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:27.885490 14755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16bcfad782e545febf5ab8a8e05488f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:27.885828 14516 tablet_service.cc:1467] Processing CreateTablet for tablet 16bcfad782e545febf5ab8a8e05488f4 (DEFAULT_TABLE table=test-table [id=06b98188a49948fa924827666bfd21e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:14:27.886121 14516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16bcfad782e545febf5ab8a8e05488f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:27.887840 15132 tablet_bootstrap.cc:492] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f: Bootstrap starting.
I20260809 23:14:27.888386 15137 tablet_bootstrap.cc:492] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39: Bootstrap starting.
I20260809 23:14:27.888463 15132 tablet_bootstrap.cc:654] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:27.888844 15132 log.cc:824] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:27.889036 15137 tablet_bootstrap.cc:654] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:27.889081 15139 tablet_bootstrap.cc:492] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551: Bootstrap starting.
I20260809 23:14:27.889407 15137 log.cc:824] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:27.889546 15132 tablet_bootstrap.cc:492] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f: No bootstrap required, opened a new log
I20260809 23:14:27.889613 15132 ts_tablet_manager.cc:1397] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:14:27.889780 15139 tablet_bootstrap.cc:654] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:27.890118 15139 log.cc:824] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:27.890314 15137 tablet_bootstrap.cc:492] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39: No bootstrap required, opened a new log
I20260809 23:14:27.890429 15137 ts_tablet_manager.cc:1397] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:27.890782 15139 tablet_bootstrap.cc:492] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551: No bootstrap required, opened a new log
I20260809 23:14:27.890863 15139 ts_tablet_manager.cc:1397] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:14:27.892338 15139 raft_consensus.cc:348] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66402c0016014fbbbc0e6c094ea5b551" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40739 } } peers { permanent_uuid: "fbd85ed7dd8d47058f5e51279d68ed39" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33227 } } peers { permanent_uuid: "3d67646499fc4284afd0d289a2c9c04f" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34053 } }
I20260809 23:14:27.892333 15132 raft_consensus.cc:348] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66402c0016014fbbbc0e6c094ea5b551" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40739 } } peers { permanent_uuid: "fbd85ed7dd8d47058f5e51279d68ed39" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33227 } } peers { permanent_uuid: "3d67646499fc4284afd0d289a2c9c04f" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34053 } }
I20260809 23:14:27.892488 15132 raft_consensus.cc:374] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:27.892488 15139 raft_consensus.cc:374] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:27.892539 15139 raft_consensus.cc:729] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66402c0016014fbbbc0e6c094ea5b551, State: Initialized, Role: FOLLOWER
I20260809 23:14:27.892539 15132 raft_consensus.cc:729] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d67646499fc4284afd0d289a2c9c04f, State: Initialized, Role: FOLLOWER
I20260809 23:14:27.892635 15139 consensus_queue.cc:260] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66402c0016014fbbbc0e6c094ea5b551" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40739 } } peers { permanent_uuid: "fbd85ed7dd8d47058f5e51279d68ed39" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33227 } } peers { permanent_uuid: "3d67646499fc4284afd0d289a2c9c04f" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34053 } }
I20260809 23:14:27.892658 15132 consensus_queue.cc:260] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66402c0016014fbbbc0e6c094ea5b551" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40739 } } peers { permanent_uuid: "fbd85ed7dd8d47058f5e51279d68ed39" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33227 } } peers { permanent_uuid: "3d67646499fc4284afd0d289a2c9c04f" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34053 } }
I20260809 23:14:27.892881 15132 ts_tablet_manager.cc:1428] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 23:14:27.892901 15139 ts_tablet_manager.cc:1428] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:14:27.892982 14627 heartbeater.cc:503] Master 127.2.76.254:37751 was elected leader, sending a full tablet report...
I20260809 23:14:27.892884 15137 raft_consensus.cc:348] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66402c0016014fbbbc0e6c094ea5b551" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40739 } } peers { permanent_uuid: "fbd85ed7dd8d47058f5e51279d68ed39" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33227 } } peers { permanent_uuid: "3d67646499fc4284afd0d289a2c9c04f" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34053 } }
I20260809 23:14:27.893172 14859 heartbeater.cc:503] Master 127.2.76.254:37751 was elected leader, sending a full tablet report...
I20260809 23:14:27.893209 15137 raft_consensus.cc:374] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:27.893244 15137 raft_consensus.cc:729] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd85ed7dd8d47058f5e51279d68ed39, State: Initialized, Role: FOLLOWER
I20260809 23:14:27.893728 15137 consensus_queue.cc:260] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66402c0016014fbbbc0e6c094ea5b551" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40739 } } peers { permanent_uuid: "fbd85ed7dd8d47058f5e51279d68ed39" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33227 } } peers { permanent_uuid: "3d67646499fc4284afd0d289a2c9c04f" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34053 } }
I20260809 23:14:27.894279 15137 ts_tablet_manager.cc:1428] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 23:14:27.894622 15091 heartbeater.cc:503] Master 127.2.76.254:37751 was elected leader, sending a full tablet report...
I20260809 23:14:27.960770 15145 raft_consensus.cc:482] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:14:27.960868 15145 raft_consensus.cc:504] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66402c0016014fbbbc0e6c094ea5b551" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40739 } } peers { permanent_uuid: "fbd85ed7dd8d47058f5e51279d68ed39" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33227 } } peers { permanent_uuid: "3d67646499fc4284afd0d289a2c9c04f" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34053 } }
I20260809 23:14:27.961289 15145 leader_election.cc:288] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd85ed7dd8d47058f5e51279d68ed39 (127.2.76.195:33227), 3d67646499fc4284afd0d289a2c9c04f (127.2.76.194:34053)
I20260809 23:14:27.963627 14633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.961430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:33227 (local address 127.2.76.193:38333)
0809 23:14:27.961483 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.961493 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.961538 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.961671 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.961672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.961674 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:27.961743 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.961744 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.962334 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.962335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.962956 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.962958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.963400 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.963401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.963461 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.963464 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.963523 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.963562 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":23}
I20260809 23:14:27.963618 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.961509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:38333 (local address 127.2.76.195:33227)
0809 23:14:27.961543 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.961544 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.961569 (+ 25us) server_negotiation.cc:407] Connection header received
0809 23:14:27.961591 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.961592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.961601 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.961649 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:27.961825 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.962253 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.963041 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.963326 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.963399 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.963407 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.963539 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.963566 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:14:27.963711 15129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.961535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:40485 (local address 127.2.76.194:34053)
0809 23:14:27.961576 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.961578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.961586 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:27.961604 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.961604 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.961612 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.961658 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:14:27.961924 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.962365 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.963113 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.963408 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.963483 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.963489 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.963626 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.963658 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 23:14:27.963727 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.961430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:34053 (local address 127.2.76.193:40485)
0809 23:14:27.961520 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:27.961529 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:27.961558 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:27.961671 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:27.961672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:27.961675 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:14:27.961844 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.961845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.962436 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.962437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.963033 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:27.963034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.963482 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.963484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:27.963559 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.963561 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:14:27.963619 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 23:14:27.963655 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":50}
I20260809 23:14:27.963986 14788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bcfad782e545febf5ab8a8e05488f4" candidate_uuid: "66402c0016014fbbbc0e6c094ea5b551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d67646499fc4284afd0d289a2c9c04f" is_pre_election: true
I20260809 23:14:27.964063 15011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bcfad782e545febf5ab8a8e05488f4" candidate_uuid: "66402c0016014fbbbc0e6c094ea5b551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd85ed7dd8d47058f5e51279d68ed39" is_pre_election: true
I20260809 23:14:27.964133 14788 raft_consensus.cc:2454] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66402c0016014fbbbc0e6c094ea5b551 in term 0.
I20260809 23:14:27.964179 15011 raft_consensus.cc:2454] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66402c0016014fbbbc0e6c094ea5b551 in term 0.
I20260809 23:14:27.964367 14443 leader_election.cc:302] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d67646499fc4284afd0d289a2c9c04f, 66402c0016014fbbbc0e6c094ea5b551; no voters:
I20260809 23:14:27.964527 15145 raft_consensus.cc:2781] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:14:27.964562 15145 raft_consensus.cc:482] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:14:27.964576 15145 raft_consensus.cc:3037] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:27.965183 15145 raft_consensus.cc:504] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66402c0016014fbbbc0e6c094ea5b551" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40739 } } peers { permanent_uuid: "fbd85ed7dd8d47058f5e51279d68ed39" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33227 } } peers { permanent_uuid: "3d67646499fc4284afd0d289a2c9c04f" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34053 } }
I20260809 23:14:27.965308 15145 leader_election.cc:288] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [CANDIDATE]: Term 1 election: Requested vote from peers fbd85ed7dd8d47058f5e51279d68ed39 (127.2.76.195:33227), 3d67646499fc4284afd0d289a2c9c04f (127.2.76.194:34053)
I20260809 23:14:27.965432 15011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bcfad782e545febf5ab8a8e05488f4" candidate_uuid: "66402c0016014fbbbc0e6c094ea5b551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd85ed7dd8d47058f5e51279d68ed39"
I20260809 23:14:27.965502 15011 raft_consensus.cc:3037] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:27.965457 14788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16bcfad782e545febf5ab8a8e05488f4" candidate_uuid: "66402c0016014fbbbc0e6c094ea5b551" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d67646499fc4284afd0d289a2c9c04f"
I20260809 23:14:27.965545 14788 raft_consensus.cc:3037] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:27.966042 15011 raft_consensus.cc:2454] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66402c0016014fbbbc0e6c094ea5b551 in term 1.
I20260809 23:14:27.966215 14442 leader_election.cc:302] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66402c0016014fbbbc0e6c094ea5b551, fbd85ed7dd8d47058f5e51279d68ed39; no voters:
I20260809 23:14:27.966250 14788 raft_consensus.cc:2454] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66402c0016014fbbbc0e6c094ea5b551 in term 1.
I20260809 23:14:27.966307 15145 raft_consensus.cc:2781] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:27.966440 15145 raft_consensus.cc:686] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [term 1 LEADER]: Becoming Leader. State: Replica: 66402c0016014fbbbc0e6c094ea5b551, State: Running, Role: LEADER
I20260809 23:14:27.966567 15145 consensus_queue.cc:237] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66402c0016014fbbbc0e6c094ea5b551" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40739 } } peers { permanent_uuid: "fbd85ed7dd8d47058f5e51279d68ed39" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33227 } } peers { permanent_uuid: "3d67646499fc4284afd0d289a2c9c04f" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34053 } }
I20260809 23:14:27.967221 12933 catalog_manager.cc:5495] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66402c0016014fbbbc0e6c094ea5b551 (127.2.76.193). New cstate: current_term: 1 leader_uuid: "66402c0016014fbbbc0e6c094ea5b551" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66402c0016014fbbbc0e6c094ea5b551" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbd85ed7dd8d47058f5e51279d68ed39" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d67646499fc4284afd0d289a2c9c04f" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34053 } health_report { overall_health: UNKNOWN } } }
I20260809 23:14:27.976855 15126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.974524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35020 (local address 127.2.76.193:40739)
0809 23:14:27.974575 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.974576 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.974584 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:14:27.974610 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.974611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.974626 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.974705 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:14:27.974827 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.975256 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.976025 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:27.976293 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:27.976378 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:27.976569 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:14:27.976661 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:14:27.976737 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:27.976774 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 23:14:27.976793 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 23:14:27.978374 15011 raft_consensus.cc:1261] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39 [term 1 FOLLOWER]: Refusing update from remote peer 66402c0016014fbbbc0e6c094ea5b551: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:27.978425 14788 raft_consensus.cc:1261] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f [term 1 FOLLOWER]: Refusing update from remote peer 66402c0016014fbbbc0e6c094ea5b551: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:14:27.978736 15145 consensus_queue.cc:1035] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd85ed7dd8d47058f5e51279d68ed39" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:14:27.978855 15150 consensus_queue.cc:1035] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d67646499fc4284afd0d289a2c9c04f" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 23:14:27.980331 14862 tablet.cc:2369] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:14:27.981618 15157 mvcc.cc:205] Tried to move back new op lower bound from 7316755529636929536 to 7316755529591459840. Current Snapshot: MvccSnapshot[applied={T|T < 7316755529636929536}]
I20260809 23:14:27.981678 15153 mvcc.cc:205] Tried to move back new op lower bound from 7316755529636929536 to 7316755529591459840. Current Snapshot: MvccSnapshot[applied={T|T < 7316755529636929536}]
I20260809 23:14:27.982858 15156 mvcc.cc:205] Tried to move back new op lower bound from 7316755529636929536 to 7316755529591459840. Current Snapshot: MvccSnapshot[applied={T|T < 7316755529636929536}]
I20260809 23:14:27.984258 12927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:27.984350 12927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:14:27.985574 12927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60c9c82a25364f28818b186daee44f4c: Sending DeleteTablet for 3 replicas of tablet 16bcfad782e545febf5ab8a8e05488f4
I20260809 23:14:27.985911 14755 tablet_service.cc:1514] Processing DeleteTablet for tablet 16bcfad782e545febf5ab8a8e05488f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:27 UTC) from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:48838
I20260809 23:14:27.985917 14979 tablet_service.cc:1514] Processing DeleteTablet for tablet 16bcfad782e545febf5ab8a8e05488f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:27 UTC) from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:51942
I20260809 23:14:27.986344 15177 tablet_replica.cc:330] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39: stopping tablet replica
I20260809 23:14:27.986490 15177 raft_consensus.cc:2229] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:27.986594 15177 raft_consensus.cc:2258] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:27.986624 14516 tablet_service.cc:1514] Processing DeleteTablet for tablet 16bcfad782e545febf5ab8a8e05488f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:14:27 UTC) from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:35008
I20260809 23:14:27.986723 15180 tablet_replica.cc:330] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f: stopping tablet replica
I20260809 23:14:27.986814 15180 raft_consensus.cc:2229] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:14:27.986891 15180 raft_consensus.cc:2258] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:27.987286 15180 ts_tablet_manager.cc:1905] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:27.987291 15190 tablet_replica.cc:330] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551: stopping tablet replica
I20260809 23:14:27.987361 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 12872
I20260809 23:14:27.987382 15190 raft_consensus.cc:2229] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:14:27.995716 15190 raft_consensus.cc:2258] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:14:27.996181 15190 ts_tablet_manager.cc:1905] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:14:27.997176 15180 ts_tablet_manager.cc:1918] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:14:27.997248 15180 log.cc:1196] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/16bcfad782e545febf5ab8a8e05488f4
I20260809 23:14:27.997525 15180 ts_tablet_manager.cc:1939] T 16bcfad782e545febf5ab8a8e05488f4 P 3d67646499fc4284afd0d289a2c9c04f: Deleting consensus metadata
I20260809 23:14:27.998219 15190 ts_tablet_manager.cc:1918] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:14:27.998277 15190 log.cc:1196] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-0/wal/wals/16bcfad782e545febf5ab8a8e05488f4
I20260809 23:14:27.998498 15190 ts_tablet_manager.cc:1939] T 16bcfad782e545febf5ab8a8e05488f4 P 66402c0016014fbbbc0e6c094ea5b551: Deleting consensus metadata
I20260809 23:14:28.003180 15177 ts_tablet_manager.cc:1905] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 23:14:28.004616 14671 connection.cc:535] server connection from 127.0.0.1:48838 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 23:14:28.005009 14445 connection.cc:535] server connection from 127.0.0.1:35008 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 23:14:28.005323 15177 ts_tablet_manager.cc:1918] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:14:28.005391 15177 log.cc:1196] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/16bcfad782e545febf5ab8a8e05488f4
I20260809 23:14:28.005646 15177 ts_tablet_manager.cc:1939] T 16bcfad782e545febf5ab8a8e05488f4 P fbd85ed7dd8d47058f5e51279d68ed39: Deleting consensus metadata
W20260809 23:14:28.006000 14903 connection.cc:441] server connection from 127.0.0.1:51942 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51942 (request call id 1) could send its response
I20260809 23:14:28.006924 15120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:27.986944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47042 (local address 127.2.76.252:40671)
0809 23:14:27.986985 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:27.986987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:27.998953 (+ 11966us) server_negotiation.cc:407] Connection header received
0809 23:14:27.999064 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:27.999065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:27.999079 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:27.999108 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:27.999951 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:28.000793 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:28.001872 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:28.002211 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:28.002251 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:28.003905 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:28.003907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:28.003907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:28.003914 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:28.005732 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:28.006149 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:28.006150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:28.006151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:28.006208 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:28.006402 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:28.006402 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:28.006403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:28.006519 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:28.006630 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:28.006751 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 23:14:28.006826 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:14:28.014436 15120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:28.007221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47058 (local address 127.2.76.252:40671)
0809 23:14:28.007257 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:28.007258 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:28.007417 (+ 159us) server_negotiation.cc:407] Connection header received
0809 23:14:28.007488 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:28.007489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:28.007507 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:28.007548 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:28.008056 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:28.008677 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:28.009731 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:28.009954 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:28.010026 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:28.011043 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:28.011044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:28.011045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:28.011051 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:28.012621 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:28.012999 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:28.013000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:28.013000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:28.013057 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:28.013272 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:28.013273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:28.013274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:28.013399 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:28.013501 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:28.014287 (+ 786us) server_negotiation.cc:299] Negotiation successful
0809 23:14:28.014353 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:14:28.015275 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:28.007463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.2.76.253:40323)
0809 23:14:28.007498 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:28.007500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:28.007580 (+ 80us) server_negotiation.cc:407] Connection header received
0809 23:14:28.007654 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:28.007655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:28.007668 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:28.007720 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:28.008373 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:28.008961 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:28.010850 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:28.011137 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:28.011177 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:28.012450 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:28.012452 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:28.012452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:28.012459 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:28.014016 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:28.014325 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:28.014326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:28.014327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:28.014377 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:28.014625 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:28.014626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:28.014627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:28.014766 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:28.014856 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:28.015055 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 23:14:28.015151 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:14:28.984182 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:28.983834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:41295)
0809 23:14:28.984032 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:28.984128 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":72,"threads_started":1}
I20260809 23:14:28.984244 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:28.983909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:56323)
0809 23:14:28.984128 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:28.984196 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":53,"threads_started":1}
W20260809 23:14:28.984457 14859 heartbeater.cc:644] Failed to heartbeat to 127.2.76.254:37751 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
I20260809 23:14:28.984514 15204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:28.984014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:48239)
0809 23:14:28.984409 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:28.984473 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":43,"threads_started":1}
W20260809 23:14:28.984557 15091 heartbeater.cc:644] Failed to heartbeat to 127.2.76.254:37751 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
W20260809 23:14:28.984642 14627 heartbeater.cc:644] Failed to heartbeat to 127.2.76.254:37751 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
I20260809 23:14:29.486347 15206 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60c9c82a25364f28818b186daee44f4c)
I20260809 23:14:29.486500 15206 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } }
I20260809 23:14:29.486763 15206 leader_election.cc:288] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60c9c82a25364f28818b186daee44f4c (127.2.76.254:37751)
I20260809 23:14:29.487056 15209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:29.486869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:41392)
0809 23:14:29.486965 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:29.487018 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":58,"threads_started":1}
W20260809 23:14:29.487175 13975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
I20260809 23:14:29.487537 15209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:29.487438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:41406)
0809 23:14:29.487484 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:29.487519 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 23:14:29.487629 13975 leader_election.cc:334] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60c9c82a25364f28818b186daee44f4c (127.2.76.254:37751): Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
I20260809 23:14:29.487703 13975 leader_election.cc:302] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ea2558e1baef47b2b24b355d6270d1e3; no voters: 60c9c82a25364f28818b186daee44f4c
I20260809 23:14:29.487807 15206 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:14:29.984993 15217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:29.984816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:58493)
0809 23:14:29.984913 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:29.984944 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":52,"threads_started":1}
I20260809 23:14:29.984997 15216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:29.984751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:41437)
0809 23:14:29.984910 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:29.984941 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":62,"threads_started":1}
I20260809 23:14:29.985044 15218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:29.984871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:60337)
0809 23:14:29.984969 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:29.985004 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":47,"threads_started":1}
I20260809 23:14:30.985641 15220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:30.985408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:33419)
0809 23:14:30.985537 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:30.985579 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260809 23:14:30.985750 15219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:30.985397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:53357)
0809 23:14:30.985674 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:30.985709 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":71,"threads_started":1}
I20260809 23:14:30.985757 15222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:30.985538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:40847)
0809 23:14:30.985661 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:30.985693 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260809 23:14:31.664196 15223 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60c9c82a25364f28818b186daee44f4c)
I20260809 23:14:31.664299 15223 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } }
I20260809 23:14:31.664455 15223 leader_election.cc:288] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60c9c82a25364f28818b186daee44f4c (127.2.76.254:37751)
I20260809 23:14:31.664708 15224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:31.664547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:41688)
0809 23:14:31.664643 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:31.664674 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":45,"threads_started":1}
I20260809 23:14:31.665098 15224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:31.665015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:41692)
0809 23:14:31.665056 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:31.665081 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 23:14:31.665172 13975 leader_election.cc:334] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60c9c82a25364f28818b186daee44f4c (127.2.76.254:37751): Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
I20260809 23:14:31.665227 13975 leader_election.cc:302] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ea2558e1baef47b2b24b355d6270d1e3; no voters: 60c9c82a25364f28818b186daee44f4c
I20260809 23:14:31.665305 15223 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:14:31.986303 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:31.986073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:56387)
0809 23:14:31.986225 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:31.986258 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260809 23:14:31.986304 15226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:31.986074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:55813)
0809 23:14:31.986220 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:31.986263 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260809 23:14:31.986521 15227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:31.986093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:38097)
0809 23:14:31.986451 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:31.986485 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":52,"threads_started":1}
I20260809 23:14:32.986961 15232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:32.986721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:54975)
0809 23:14:32.986864 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:32.986903 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260809 23:14:32.987042 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:32.986855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:52227)
0809 23:14:32.986971 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:32.987000 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":55,"threads_started":1}
I20260809 23:14:32.987030 15233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:32.986852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:40335)
0809 23:14:32.986951 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:32.986985 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":62,"threads_started":1}
I20260809 23:14:33.987579 15237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:33.987354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:41877)
0809 23:14:33.987496 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:33.987536 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260809 23:14:33.987582 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:33.987407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:45493)
0809 23:14:33.987499 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:33.987536 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":41,"threads_started":1}
I20260809 23:14:33.987582 15238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:33.987367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:38547)
0809 23:14:33.987499 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:33.987532 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":57,"threads_started":1}
I20260809 23:14:34.287148 15242 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60c9c82a25364f28818b186daee44f4c)
I20260809 23:14:34.287235 15242 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } }
I20260809 23:14:34.287338 15242 leader_election.cc:288] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60c9c82a25364f28818b186daee44f4c (127.2.76.254:37751)
I20260809 23:14:34.287788 15243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:34.287486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:41936)
0809 23:14:34.287702 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:34.287750 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":166,"threads_started":1}
I20260809 23:14:34.288240 15243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:34.288143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:41944)
0809 23:14:34.288191 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:34.288220 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 23:14:34.288343 13975 leader_election.cc:334] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60c9c82a25364f28818b186daee44f4c (127.2.76.254:37751): Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
I20260809 23:14:34.288403 13975 leader_election.cc:302] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ea2558e1baef47b2b24b355d6270d1e3; no voters: 60c9c82a25364f28818b186daee44f4c
I20260809 23:14:34.288515 15242 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:14:34.988147 15247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:34.987939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:51961)
0809 23:14:34.988065 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:34.988094 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":54,"threads_started":1}
I20260809 23:14:34.988148 15246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:34.987925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:41923)
0809 23:14:34.988040 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:34.988077 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":64,"threads_started":1}
I20260809 23:14:34.988173 15250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:34.987979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:37753)
0809 23:14:34.988070 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:34.988127 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":40,"threads_started":1}
I20260809 23:14:35.988763 15252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:35.988562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:60853)
0809 23:14:35.988678 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:35.988710 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260809 23:14:35.988759 15251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:35.988554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:53349)
0809 23:14:35.988677 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:35.988712 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260809 23:14:35.988759 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:35.988610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:59211)
0809 23:14:35.988694 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:35.988722 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70,"thread_start_us":41,"threads_started":1}
I20260809 23:14:36.950726 15257 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60c9c82a25364f28818b186daee44f4c)
I20260809 23:14:36.950851 15257 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } }
I20260809 23:14:36.951023 15257 leader_election.cc:288] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60c9c82a25364f28818b186daee44f4c (127.2.76.254:37751)
I20260809 23:14:36.951318 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:36.951140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:42146)
0809 23:14:36.951251 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:36.951283 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":54,"threads_started":1}
W20260809 23:14:36.951418 13975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260809 23:14:36.951792 15259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:36.951696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:42154)
0809 23:14:36.951747 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:36.951772 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 23:14:36.951862 13975 leader_election.cc:334] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60c9c82a25364f28818b186daee44f4c (127.2.76.254:37751): Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
I20260809 23:14:36.951925 13975 leader_election.cc:302] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ea2558e1baef47b2b24b355d6270d1e3; no voters: 60c9c82a25364f28818b186daee44f4c
I20260809 23:14:36.952008 15257 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:14:36.989300 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:36.989126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:44013)
0809 23:14:36.989241 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:36.989267 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":67,"threads_started":1}
I20260809 23:14:36.989295 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:36.989099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:43345)
0809 23:14:36.989217 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:36.989247 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260809 23:14:36.989377 15267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:36.989204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:54393)
0809 23:14:36.989311 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:36.989338 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260809 23:14:37.989912 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:37.989683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:48631)
0809 23:14:37.989822 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:37.989862 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260809 23:14:37.989948 15275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:37.989776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:44925)
0809 23:14:37.989886 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:37.989915 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":54,"threads_started":1}
I20260809 23:14:37.989948 15272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:37.989764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:59463)
0809 23:14:37.989868 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:37.989903 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":60,"threads_started":1}
I20260809 23:14:38.990494 15276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:38.990267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:47851)
0809 23:14:38.990410 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:38.990447 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260809 23:14:38.990502 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:38.990292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:34395)
0809 23:14:38.990409 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:38.990450 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260809 23:14:38.990553 15281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:38.990371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:50793)
0809 23:14:38.990473 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:38.990511 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":50,"threads_started":1}
I20260809 23:14:39.991178 15284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:39.990964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:55141)
0809 23:14:39.991100 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:39.991131 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":58,"threads_started":1}
I20260809 23:14:39.991231 15285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:39.990964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:43189)
0809 23:14:39.991114 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:39.991153 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260809 23:14:39.991261 15283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:39.990938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:56095)
0809 23:14:39.991069 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:39.991113 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260809 23:14:40.991892 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:40.991653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:38875)
0809 23:14:40.991789 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:40.991829 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260809 23:14:40.991910 15290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:40.991709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:50851)
0809 23:14:40.991830 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:40.991863 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260809 23:14:40.991910 15291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:40.991719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:36717)
0809 23:14:40.991834 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:40.991863 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":54,"threads_started":1}
I20260809 23:14:41.992496 15292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:41.992275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:33851)
0809 23:14:41.992403 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:41.992444 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260809 23:14:41.992496 15293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:41.992274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:47469)
0809 23:14:41.992407 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:41.992442 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260809 23:14:41.992605 15294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:41.992417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:50845)
0809 23:14:41.992539 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:41.992567 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":51,"threads_started":1}
I20260809 23:14:42.993247 15298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:42.993023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:53685)
0809 23:14:42.993165 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:42.993192 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260809 23:14:42.993265 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:42.993012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:44517)
0809 23:14:42.993144 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:42.993184 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":58,"threads_started":1}
I20260809 23:14:42.993323 15295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:42.992995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:56771)
0809 23:14:42.993147 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:42.993277 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260809 23:14:43.576694 15299 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60c9c82a25364f28818b186daee44f4c)
I20260809 23:14:43.576778 15299 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } }
I20260809 23:14:43.576931 15299 leader_election.cc:288] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60c9c82a25364f28818b186daee44f4c (127.2.76.254:37751)
I20260809 23:14:43.577255 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:43.577088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:41480)
0809 23:14:43.577180 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:43.577219 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":53,"threads_started":1}
W20260809 23:14:43.577329 13975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
I20260809 23:14:43.577652 15300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:43.577569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:41482)
0809 23:14:43.577612 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:43.577636 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 23:14:43.577742 13975 leader_election.cc:334] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60c9c82a25364f28818b186daee44f4c (127.2.76.254:37751): Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
I20260809 23:14:43.577811 13975 leader_election.cc:302] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ea2558e1baef47b2b24b355d6270d1e3; no voters: 60c9c82a25364f28818b186daee44f4c
I20260809 23:14:43.577890 15299 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:14:43.993961 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:43.993643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:45629)
0809 23:14:43.993844 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:43.993901 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":72,"threads_started":1}
I20260809 23:14:43.993961 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:43.993733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:44049)
0809 23:14:43.993867 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:43.993906 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260809 23:14:43.994164 15309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:43.993961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:47655)
0809 23:14:43.994096 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:43.994124 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260809 23:14:44.994606 15311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:44.994382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:46037)
0809 23:14:44.994505 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:44.994546 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260809 23:14:44.994693 15312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:44.994497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:58995)
0809 23:14:44.994609 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:44.994645 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":67,"threads_started":1}
I20260809 23:14:44.994735 15313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:44.994570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:59595)
0809 23:14:44.994665 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:44.994694 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":53,"threads_started":1}
I20260809 23:14:45.995307 15314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:45.995057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:49815)
0809 23:14:45.995206 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:45.995247 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260809 23:14:45.995450 15318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:45.995261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:58909)
0809 23:14:45.995372 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:45.995403 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":54,"threads_started":1}
I20260809 23:14:45.995645 15317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:45.995267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:54081)
0809 23:14:45.995565 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:45.995597 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":45,"threads_started":1}
I20260809 23:14:46.996009 15320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:46.995800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:48229)
0809 23:14:46.995925 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:46.995958 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260809 23:14:46.995986 15319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:46.995782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:44077)
0809 23:14:46.995894 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:46.995928 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":68,"threads_started":1}
I20260809 23:14:46.996197 15321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:46.995992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:51033)
0809 23:14:46.996119 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:46.996155 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":49,"threads_started":1}
I20260809 23:14:47.544986 15322 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 60c9c82a25364f28818b186daee44f4c)
I20260809 23:14:47.545066 15322 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "60c9c82a25364f28818b186daee44f4c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37751 } } peers { permanent_uuid: "ea2558e1baef47b2b24b355d6270d1e3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 40323 } attrs { promote: false } } peers { permanent_uuid: "ce0985f9218f49629769fe7d73a8e43c" member_type: NON_VOTER last_known_addr { host: "127.2.76.252" port: 40671 } attrs { promote: true } }
I20260809 23:14:47.545207 15322 leader_election.cc:288] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60c9c82a25364f28818b186daee44f4c (127.2.76.254:37751)
I20260809 23:14:47.545485 15323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:47.545318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:41688)
0809 23:14:47.545411 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:47.545442 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":56,"threads_started":1}
I20260809 23:14:47.545833 15323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:47.545745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.0.0.1:41692)
0809 23:14:47.545790 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:47.545813 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 23:14:47.545913 13975 leader_election.cc:334] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 60c9c82a25364f28818b186daee44f4c (127.2.76.254:37751): Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
I20260809 23:14:47.545954 13975 leader_election.cc:302] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: ea2558e1baef47b2b24b355d6270d1e3; no voters: 60c9c82a25364f28818b186daee44f4c
I20260809 23:14:47.546028 15322 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ea2558e1baef47b2b24b355d6270d1e3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:14:47.996637 15325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:47.996409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:42485)
0809 23:14:47.996534 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:47.996579 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":67,"threads_started":1}
I20260809 23:14:47.996676 15326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:47.996433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:42265)
0809 23:14:47.996603 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:47.996631 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":71,"threads_started":1}
I20260809 23:14:47.996824 15327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:47.996516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:39227)
0809 23:14:47.996648 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:47.996685 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260809 23:14:48.997262 15328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:48.997018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:58367)
0809 23:14:48.997162 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:48.997206 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260809 23:14:48.997301 15331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:48.997136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:47607)
0809 23:14:48.997229 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:48.997254 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":47,"threads_started":1}
I20260809 23:14:48.997301 15330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:48.997073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:55119)
0809 23:14:48.997215 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:48.997255 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260809 23:14:49.997888 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:49.997649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:50755)
0809 23:14:49.997793 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:49.997828 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260809 23:14:49.997887 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:49.997649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:50779)
0809 23:14:49.997795 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:49.997834 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260809 23:14:49.997901 15336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:49.997692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:50525)
0809 23:14:49.997839 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:49.997866 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260809 23:14:50.998520 15340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:50.998277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:42525)
0809 23:14:50.998417 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:50.998453 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260809 23:14:50.998502 15338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:50.998273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:55501)
0809 23:14:50.998413 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:50.998459 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260809 23:14:50.998665 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:50.998438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:35823)
0809 23:14:50.998554 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:50.998581 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":41,"threads_started":1}
I20260809 23:14:51.999156 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:51.998920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:60403)
0809 23:14:51.999061 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:51.999092 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":60,"threads_started":1}
I20260809 23:14:51.999155 15344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:51.998920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:38359)
0809 23:14:51.999056 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:51.999098 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260809 23:14:51.999224 15347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:51.999031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:51717)
0809 23:14:51.999153 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:51.999186 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260809 23:14:52.999795 15353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:52.999553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:38047)
0809 23:14:52.999719 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:52.999751 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260809 23:14:52.999822 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:52.999553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:33177)
0809 23:14:52.999733 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:52.999771 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":80,"threads_started":1}
I20260809 23:14:52.999966 15354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:52.999775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:36243)
0809 23:14:52.999894 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:52.999922 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":50,"threads_started":1}
I20260809 23:14:54.000439 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:54.000217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:60007)
0809 23:14:54.000340 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:54.000378 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
I20260809 23:14:54.000547 15360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:54.000368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:37881)
0809 23:14:54.000477 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:54.000506 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260809 23:14:54.000617 15359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:54.000345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:32867)
0809 23:14:54.000459 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:54.000494 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":70,"threads_started":1}
I20260809 23:14:55.001091 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:55.000879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:41783)
0809 23:14:55.000996 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:55.001035 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":66,"threads_started":1}
I20260809 23:14:55.001091 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:55.000878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:54899)
0809 23:14:55.001020 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:55.001051 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260809 23:14:55.001317 15365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:55.001039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:57341)
0809 23:14:55.001134 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:55.001273 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":55,"threads_started":1}
I20260809 23:14:56.001765 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:56.001531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:37817)
0809 23:14:56.001674 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:56.001719 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260809 23:14:56.001802 15367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:56.001572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:50281)
0809 23:14:56.001710 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:56.001752 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260809 23:14:56.001832 15369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:56.001672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:47921)
0809 23:14:56.001766 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:56.001789 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":46,"threads_started":1}
I20260809 23:14:57.002395 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:57.002170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:37285)
0809 23:14:57.002299 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:57.002335 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260809 23:14:57.002451 15375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:57.002159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:38951)
0809 23:14:57.002370 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:57.002403 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":99,"threads_started":1}
I20260809 23:14:57.002491 15379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:57.002314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:58621)
0809 23:14:57.002413 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:57.002437 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":48,"threads_started":1}
I20260809 23:14:58.003093 15387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:58.002886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.194:45001)
0809 23:14:58.003007 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:58.003041 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":58,"threads_started":1}
I20260809 23:14:58.003144 15381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:58.002847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.195:52463)
0809 23:14:58.002969 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:58.003040 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260809 23:14:58.003227 15384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:58.002847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37751 (local address 127.2.76.193:36059)
0809 23:14:58.003054 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:58.003096 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:37751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":71,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:903: Failure
Failed
Bad status: Timed out: Could not connect to the cluster: ConnectToClusterRpc(addrs: 127.2.76.254:37751,127.2.76.253:40323,127.2.76.252:40671, num_attempts: 243) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.2.76.254:37751,127.2.76.253:40323,127.2.76.252:40671, num_attempts: 1)
I20260809 23:14:58.184551 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 14264
I20260809 23:14:58.196878 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 14637
I20260809 23:14:58.210877 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 14868
I20260809 23:14:58.218987 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 13760
I20260809 23:14:58.227769 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 14099
2026-08-09T23:14:58Z chronyd exiting
I20260809 23:14:58.248633 2355 test_util.cc:175] -----------------------------------------------
I20260809 23:14:58.248677 2355 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786317238298104-2355-0
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (33094 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:14:58 dist-test-slave-7vjv krb5kdc[15391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:14:58 dist-test-slave-7vjv krb5kdc[15391](info): set up 2 sockets
Aug 09 23:14:58 dist-test-slave-7vjv krb5kdc[15391](info): commencing operation
krb5kdc: starting...
W20260809 23:14:59.382745 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:14:59 dist-test-slave-7vjv krb5kdc[15391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:14:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:14:59Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.254.keytab.
Entry for principal oryx/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.254.keytab.
I20260809 23:14:59.494309 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:38423
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.254.keytab
--principal=oryx/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:38423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:59.569003 15419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:59.569213 15419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:59.569234 15419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:59.570358 15419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:14:59.570397 15419 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 23:14:59.570411 15419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:59.570447 15419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:59.570477 15419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:14:59.570494 15419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:14:59.571830 15419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:38423
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.254.keytab
--principal=oryx/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:38423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:59.572044 15419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:59.572252 15419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:59.576539 15424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:59.576550 15425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:59.576588 15427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:59.576601 15419 server_base.cc:1034] running on GCE node
I20260809 23:14:59.577011 15419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:59.577283 15419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:59.578429 15419 hybrid_clock.cc:648] HybridClock initialized: now 1786317299578417 us; error 29 us; skew 500 ppm
Aug 09 23:14:59 dist-test-slave-7vjv krb5kdc[15391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317299, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:59.580703 15419 init.cc:377] Logged in from keytab as oryx/127.2.76.254@KRBTEST.COM (short username oryx)
I20260809 23:14:59.581073 15419 webserver.cc:460] Webserver started at http://127.2.76.254:39309/ using document root <none> and password file <none>
I20260809 23:14:59.581269 15419 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:59.581329 15419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:59.581468 15419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:59.582329 15419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "b56ddb436aea4b0694eefbeca8f07ec0"
format_stamp: "Formatted at 2026-08-09 23:14:59 on dist-test-slave-7vjv"
server_key: "d306a82186a601c07fc54d6806596c95"
server_key_iv: "d7f45e19f204595056305eec35ee9241"
server_key_version: "encryptionkey@0"
I20260809 23:14:59.582666 15419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "b56ddb436aea4b0694eefbeca8f07ec0"
format_stamp: "Formatted at 2026-08-09 23:14:59 on dist-test-slave-7vjv"
server_key: "d306a82186a601c07fc54d6806596c95"
server_key_iv: "d7f45e19f204595056305eec35ee9241"
server_key_version: "encryptionkey@0"
I20260809 23:14:59.583897 15419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:14:59.584636 15439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:59.584800 15419 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:59.584888 15419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "b56ddb436aea4b0694eefbeca8f07ec0"
format_stamp: "Formatted at 2026-08-09 23:14:59 on dist-test-slave-7vjv"
server_key: "d306a82186a601c07fc54d6806596c95"
server_key_iv: "d7f45e19f204595056305eec35ee9241"
server_key_version: "encryptionkey@0"
I20260809 23:14:59.584965 15419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:59.602427 15419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:59.604420 15419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:59.604561 15419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:59.608609 15419 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:38423
I20260809 23:14:59.608682 15544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:38423 every 8 connection(s)
I20260809 23:14:59.608947 15419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:14:59.609536 15546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:14:59.611375 15546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0: Bootstrap starting.
I20260809 23:14:59.611863 15546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0: Neither blocks nor log segments found. Creating new log.
I20260809 23:14:59.612146 15546 log.cc:824] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0: Log is configured to *not* fsync() on all Append() calls
I20260809 23:14:59.612747 15546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0: No bootstrap required, opened a new log
I20260809 23:14:59.614488 15546 raft_consensus.cc:348] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56ddb436aea4b0694eefbeca8f07ec0" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38423 } }
I20260809 23:14:59.614614 15546 raft_consensus.cc:374] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:14:59.614634 15546 raft_consensus.cc:729] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b56ddb436aea4b0694eefbeca8f07ec0, State: Initialized, Role: FOLLOWER
I20260809 23:14:59.614718 15546 consensus_queue.cc:260] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56ddb436aea4b0694eefbeca8f07ec0" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38423 } }
I20260809 23:14:59.614763 15546 raft_consensus.cc:388] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:14:59.614781 15546 raft_consensus.cc:482] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:14:59.614804 15546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:14:59.615201 15546 raft_consensus.cc:504] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56ddb436aea4b0694eefbeca8f07ec0" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38423 } }
I20260809 23:14:59.615291 15546 leader_election.cc:302] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b56ddb436aea4b0694eefbeca8f07ec0; no voters:
I20260809 23:14:59.615410 15546 leader_election.cc:288] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:14:59.615473 15549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:14:59.615622 15549 raft_consensus.cc:686] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [term 1 LEADER]: Becoming Leader. State: Replica: b56ddb436aea4b0694eefbeca8f07ec0, State: Running, Role: LEADER
I20260809 23:14:59.615679 15546 sys_catalog.cc:564] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:14:59.615712 15549 consensus_queue.cc:237] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56ddb436aea4b0694eefbeca8f07ec0" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38423 } }
I20260809 23:14:59.616153 15551 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b56ddb436aea4b0694eefbeca8f07ec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56ddb436aea4b0694eefbeca8f07ec0" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38423 } } }
I20260809 23:14:59.616166 15552 sys_catalog.cc:455] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b56ddb436aea4b0694eefbeca8f07ec0. Latest consensus state: current_term: 1 leader_uuid: "b56ddb436aea4b0694eefbeca8f07ec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b56ddb436aea4b0694eefbeca8f07ec0" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38423 } } }
I20260809 23:14:59.616255 15551 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:59.616266 15552 sys_catalog.cc:458] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0 [sys.catalog]: This master's current role is: LEADER
I20260809 23:14:59.616439 15563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:14:59.616904 15563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:14:59.618376 15563 catalog_manager.cc:1347] Generated new cluster ID: f6df1d91736b4aadb05b0e237c97e7de
I20260809 23:14:59.618436 15563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:14:59.618530 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 15419
I20260809 23:14:59.618637 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:14:59.618808 2355 external_mini_cluster.cc:1428] Setting key f92c820bac8c2bea55ef67422c7346bf
I20260809 23:14:59.623282 15563 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:14:59.624137 15563 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 23:14:59 dist-test-slave-7vjv krb5kdc[15391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
I20260809 23:14:59.631114 15587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:59.619304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36508 (local address 127.2.76.254:38423)
0809 23:14:59.619435 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:59.619442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:59.619467 (+ 25us) server_negotiation.cc:407] Connection header received
0809 23:14:59.619574 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:59.619579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:59.619629 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:59.619756 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:59.620405 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.621668 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.622541 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.623086 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.628160 (+ 5074us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:59.628240 (+ 80us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:59.628243 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:59.628243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:59.628249 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:59.629758 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:59.630108 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:59.630110 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:59.630111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:59.630195 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:59.630394 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:59.630394 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:59.630395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:59.630597 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:59.630751 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:59.630812 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 23:14:59.630940 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
I20260809 23:14:59.633018 15563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0: Generated new TSK 0
I20260809 23:14:59.633183 15563 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.193.keytab.
Entry for principal oryx/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.193.keytab.
I20260809 23:14:59.664429 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:38423
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.193.keytab
--principal=oryx/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:59.759171 15590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:59.759334 15590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:59.759354 15590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:59.760351 15590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:59.760385 15590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:59.760402 15590 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:14:59.760417 15590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:14:59.761680 15590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.193.keytab
--principal=oryx/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.76.254:38423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:59.761874 15590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:59.762095 15590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:59.766367 15600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:59.766376 15601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:59.766402 15604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:59.766568 15590 server_base.cc:1034] running on GCE node
I20260809 23:14:59.766772 15590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:59.766994 15590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:59.768148 15590 hybrid_clock.cc:648] HybridClock initialized: now 1786317299768133 us; error 44 us; skew 500 ppm
Aug 09 23:14:59 dist-test-slave-7vjv krb5kdc[15391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317299, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:59.770381 15590 init.cc:377] Logged in from keytab as oryx/127.2.76.193@KRBTEST.COM (short username oryx)
I20260809 23:14:59.770766 15590 webserver.cc:460] Webserver started at http://127.2.76.193:42451/ using document root <none> and password file <none>
I20260809 23:14:59.770972 15590 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:59.771035 15590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:59.771154 15590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:59.772140 15590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "da180222b3ef4a44885a2eb142988da1"
format_stamp: "Formatted at 2026-08-09 23:14:59 on dist-test-slave-7vjv"
server_key: "e67163920753f2a9a0e514c6ede796cf"
server_key_iv: "d72b4447cf815ca641aed3fb41a4b9ca"
server_key_version: "encryptionkey@0"
I20260809 23:14:59.772493 15590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "da180222b3ef4a44885a2eb142988da1"
format_stamp: "Formatted at 2026-08-09 23:14:59 on dist-test-slave-7vjv"
server_key: "e67163920753f2a9a0e514c6ede796cf"
server_key_iv: "d72b4447cf815ca641aed3fb41a4b9ca"
server_key_version: "encryptionkey@0"
I20260809 23:14:59.773764 15590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:14:59.774391 15614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:59.774557 15590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:14:59.774644 15590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "da180222b3ef4a44885a2eb142988da1"
format_stamp: "Formatted at 2026-08-09 23:14:59 on dist-test-slave-7vjv"
server_key: "e67163920753f2a9a0e514c6ede796cf"
server_key_iv: "d72b4447cf815ca641aed3fb41a4b9ca"
server_key_version: "encryptionkey@0"
I20260809 23:14:59.774721 15590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:59.787361 15590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:59.789125 15590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:59.789268 15590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:59.789705 15590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:59.789767 15590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:59.789805 15590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:59.789830 15590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:14:59 dist-test-slave-7vjv krb5kdc[15391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317299, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
I20260809 23:14:59.796154 15590 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:39391
I20260809 23:14:59.796222 15794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:39391 every 8 connection(s)
I20260809 23:14:59.796458 15590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:14:59.798858 15587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:59.790132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:36753 (local address 127.2.76.254:38423)
0809 23:14:59.790209 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:59.790212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:59.790466 (+ 254us) server_negotiation.cc:407] Connection header received
0809 23:14:59.790865 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:59.790867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:59.790880 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:59.790925 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:59.791691 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.792456 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.793602 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.793889 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.793956 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:59.796310 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:59.796312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:59.796312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:59.796319 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:59.797666 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:59.798027 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:59.798028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:59.798028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:59.798079 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:59.798286 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:59.798286 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:59.798286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:59.798383 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:59.798433 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:59.798653 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 23:14:59.798744 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:14:59.798890 15623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:59.790115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:38423 (local address 127.2.76.193:36753)
0809 23:14:59.790355 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:59.790396 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:59.790776 (+ 380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:59.790984 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:59.790987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:59.791341 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:59.791584 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:59.791589 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.792553 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.792554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:59.793507 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:59.793510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.793974 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.793975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:59.794103 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:59.794562 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:59.794569 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:59.796223 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:59.797739 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:59.797745 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:59.797746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:59.797955 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:59.798137 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:59.798138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:59.798138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:59.798232 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:59.798436 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:59.798441 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:59.798546 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 23:14:59.798640 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 23:14:59.798756 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":83,"threads_started":1}
I20260809 23:14:59.798906 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 15590
I20260809 23:14:59.799017 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:14:59.799187 2355 external_mini_cluster.cc:1428] Setting key cc5b49b82d79d8838acf3eecc7cdbce5
I20260809 23:14:59.799728 15796 heartbeater.cc:347] Connected to a master server at 127.2.76.254:38423
I20260809 23:14:59.799808 15796 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:59.800009 15796 heartbeater.cc:511] Master 127.2.76.254:38423 requested a full tablet report, sending...
I20260809 23:14:59.801926 15467 ts_manager.cc:194] Registered new tserver with Master: da180222b3ef4a44885a2eb142988da1 (127.2.76.193:39391)
I20260809 23:14:59.802701 15467 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:36753
I20260809 23:14:59.805636 15587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:59.800712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:38633 (local address 127.2.76.254:38423)
0809 23:14:59.800749 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:59.800750 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:59.800835 (+ 85us) server_negotiation.cc:407] Connection header received
0809 23:14:59.800878 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:59.800878 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:59.800885 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:59.800924 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:59.801236 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.801610 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.802338 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.802500 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.802570 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:59.803337 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:59.803338 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:59.803339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:59.803345 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:59.804682 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:59.804974 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:59.804975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:59.804976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:59.805037 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:59.805220 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:59.805220 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:59.805221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:59.805312 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:59.805384 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:59.805499 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 23:14:59.805564 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:14:59.805651 15805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:59.800621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:38423 (local address 127.2.76.193:38633)
0809 23:14:59.800759 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:59.800780 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:59.800833 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:59.800929 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:59.800929 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:59.801121 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:59.801177 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:59.801179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.801703 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.801704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:59.802264 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:59.802266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.802570 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.802571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:59.802632 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:59.803000 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:59.803004 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:59.803260 (+ 256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:59.804757 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:59.804758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:59.804759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:59.804909 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:59.805102 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:59.805103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:59.805103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:59.805154 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:59.805377 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:59.805378 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:59.805416 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 23:14:59.805475 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 23:14:59.805565 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for oryx/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.194.keytab.
Entry for principal oryx/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.194.keytab.
I20260809 23:14:59.822567 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:38423
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.194.keytab
--principal=oryx/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:14:59.895182 15811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:14:59.895349 15811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:14:59.895370 15811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:14:59.896378 15811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:14:59.896409 15811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:14:59.896427 15811 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:14:59.896443 15811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:14:59.897732 15811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.194.keytab
--principal=oryx/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.76.254:38423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:14:59.897888 15811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:14:59.898077 15811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:14:59.902455 15821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:14:59.902486 15817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:59.902614 15811 server_base.cc:1034] running on GCE node
W20260809 23:14:59.902455 15819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:14:59.902830 15811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:14:59.902999 15811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:14:59.904150 15811 hybrid_clock.cc:648] HybridClock initialized: now 1786317299904142 us; error 23 us; skew 500 ppm
Aug 09 23:14:59 dist-test-slave-7vjv krb5kdc[15391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317299, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:14:59.906280 15811 init.cc:377] Logged in from keytab as oryx/127.2.76.194@KRBTEST.COM (short username oryx)
I20260809 23:14:59.906675 15811 webserver.cc:460] Webserver started at http://127.2.76.194:45703/ using document root <none> and password file <none>
I20260809 23:14:59.906858 15811 fs_manager.cc:359] Metadata directory not provided
I20260809 23:14:59.906895 15811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:14:59.907033 15811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:14:59.907920 15811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "9b28c3135e8d4472b58af402abdfb79d"
format_stamp: "Formatted at 2026-08-09 23:14:59 on dist-test-slave-7vjv"
server_key: "2e18b9d0dee0e15dc1717f6d14b2d907"
server_key_iv: "573a170fa88d0c11afdaa0b07a937d9d"
server_key_version: "encryptionkey@0"
I20260809 23:14:59.908277 15811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "9b28c3135e8d4472b58af402abdfb79d"
format_stamp: "Formatted at 2026-08-09 23:14:59 on dist-test-slave-7vjv"
server_key: "2e18b9d0dee0e15dc1717f6d14b2d907"
server_key_iv: "573a170fa88d0c11afdaa0b07a937d9d"
server_key_version: "encryptionkey@0"
I20260809 23:14:59.909502 15811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:14:59.910133 15834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:59.910276 15811 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:14:59.910382 15811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "9b28c3135e8d4472b58af402abdfb79d"
format_stamp: "Formatted at 2026-08-09 23:14:59 on dist-test-slave-7vjv"
server_key: "2e18b9d0dee0e15dc1717f6d14b2d907"
server_key_iv: "573a170fa88d0c11afdaa0b07a937d9d"
server_key_version: "encryptionkey@0"
I20260809 23:14:59.910456 15811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:14:59.920534 15811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:14:59.922056 15811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:14:59.922197 15811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:14:59.922672 15811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:14:59.922725 15811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:14:59.922752 15811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:14:59.922775 15811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:14:59 dist-test-slave-7vjv krb5kdc[15391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317299, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
I20260809 23:14:59.929665 15811 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:39419
I20260809 23:14:59.929736 16033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:39419 every 8 connection(s)
I20260809 23:14:59.930056 15811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:14:59.931624 15587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:59.922962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:49959 (local address 127.2.76.254:38423)
0809 23:14:59.923025 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:59.923027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:59.923178 (+ 151us) server_negotiation.cc:407] Connection header received
0809 23:14:59.923562 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:59.923563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:59.923575 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:59.923660 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:59.924237 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.925050 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.925973 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.926270 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.926340 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:59.928986 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:59.928987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:59.928988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:59.928994 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:59.930426 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:59.930764 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:59.930765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:59.930765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:59.930819 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:59.931055 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:59.931056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:59.931056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:59.931161 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:59.931229 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:59.931463 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 23:14:59.931547 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:14:59.931737 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:59.922949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:38423 (local address 127.2.76.194:49959)
0809 23:14:59.923065 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:59.923095 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:59.923458 (+ 363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:59.923720 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:59.923723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:59.923922 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:59.924072 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:59.924076 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.925156 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.925157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:59.925889 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:59.925891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.926350 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.926351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:59.926460 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:59.926887 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:59.926898 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:59.928886 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:59.930496 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:59.930500 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:59.930502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:59.930692 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:59.930885 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:59.930886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:59.930886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:59.930986 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:59.931222 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:59.931228 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:59.931345 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 23:14:59.931432 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 23:14:59.931558 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":69,"threads_started":1}
I20260809 23:14:59.932165 16036 heartbeater.cc:347] Connected to a master server at 127.2.76.254:38423
I20260809 23:14:59.932242 16036 heartbeater.cc:464] Registering TS with master...
I20260809 23:14:59.932400 16036 heartbeater.cc:511] Master 127.2.76.254:38423 requested a full tablet report, sending...
I20260809 23:14:59.932781 15463 ts_manager.cc:194] Registered new tserver with Master: 9b28c3135e8d4472b58af402abdfb79d (127.2.76.194:39419)
I20260809 23:14:59.933207 15463 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:49959
I20260809 23:14:59.936293 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 15811
I20260809 23:14:59.936410 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:14:59.936607 2355 external_mini_cluster.cc:1428] Setting key 043293faf4cacb77eb5b55473e98f32d
I20260809 23:14:59.938002 15587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:59.933347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:39847 (local address 127.2.76.254:38423)
0809 23:14:59.933483 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 23:14:59.933485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:14:59.933490 (+ 5us) server_negotiation.cc:407] Connection header received
0809 23:14:59.933509 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:14:59.933510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:14:59.933517 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:14:59.933572 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:14:59.933862 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.934247 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.934866 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.935040 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.935108 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:59.935810 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:14:59.935811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:14:59.935811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:14:59.935817 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:14:59.937020 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:59.937371 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:59.937372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:59.937372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:59.937420 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:14:59.937597 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:14:59.937597 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:14:59.937597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:14:59.937706 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:14:59.937751 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 23:14:59.937884 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 23:14:59.937937 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260809 23:14:59.938136 16046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:14:59.933054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:38423 (local address 127.2.76.194:39847)
0809 23:14:59.933181 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 23:14:59.933190 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:14:59.933235 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:14:59.933591 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:14:59.933592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:14:59.933746 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:14:59.933787 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:59.933788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.934329 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.934330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:59.934778 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:14:59.934780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:14:59.935109 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:14:59.935110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:14:59.935168 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:14:59.935484 (+ 316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:14:59.935488 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:14:59.935721 (+ 233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:14:59.937107 (+ 1386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:59.937107 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:59.937108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:59.937305 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:59.937482 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:14:59.937483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:14:59.937484 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:14:59.937547 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:14:59.937762 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:14:59.937762 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:14:59.937802 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 23:14:59.937871 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 23:14:59.937971 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for oryx/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.195.keytab.
Entry for principal oryx/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.195.keytab.
I20260809 23:14:59.960741 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:38423
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.195.keytab
--principal=oryx/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:15:00.035466 16051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:00.035635 16051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:00.035657 16051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:00.036691 16051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:00.036729 16051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:00.036746 16051 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:15:00.036761 16051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:15:00.038074 16051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.195.keytab
--principal=oryx/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.76.254:38423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:00.038244 16051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:00.038427 16051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:00.042865 16063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:00.042876 16066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:00.042906 16062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:00.043107 16051 server_base.cc:1034] running on GCE node
I20260809 23:15:00.043365 16051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:00.043625 16051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:00.044780 16051 hybrid_clock.cc:648] HybridClock initialized: now 1786317300044775 us; error 34 us; skew 500 ppm
Aug 09 23:15:00 dist-test-slave-7vjv krb5kdc[15391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317300, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:15:00.047075 16051 init.cc:377] Logged in from keytab as oryx/127.2.76.195@KRBTEST.COM (short username oryx)
I20260809 23:15:00.047552 16051 webserver.cc:460] Webserver started at http://127.2.76.195:42277/ using document root <none> and password file <none>
I20260809 23:15:00.047775 16051 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:00.047842 16051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:00.047976 16051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:00.048929 16051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "45b084e5df8b4f5aa9b271e5fd63a74c"
format_stamp: "Formatted at 2026-08-09 23:15:00 on dist-test-slave-7vjv"
server_key: "7716361183be14954da85e6974667ccc"
server_key_iv: "99a15f676026838b76887578f6578a0f"
server_key_version: "encryptionkey@0"
I20260809 23:15:00.049377 16051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "45b084e5df8b4f5aa9b271e5fd63a74c"
format_stamp: "Formatted at 2026-08-09 23:15:00 on dist-test-slave-7vjv"
server_key: "7716361183be14954da85e6974667ccc"
server_key_iv: "99a15f676026838b76887578f6578a0f"
server_key_version: "encryptionkey@0"
I20260809 23:15:00.050619 16051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:15:00.051229 16078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:00.051397 16051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:15:00.051494 16051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "45b084e5df8b4f5aa9b271e5fd63a74c"
format_stamp: "Formatted at 2026-08-09 23:15:00 on dist-test-slave-7vjv"
server_key: "7716361183be14954da85e6974667ccc"
server_key_iv: "99a15f676026838b76887578f6578a0f"
server_key_version: "encryptionkey@0"
I20260809 23:15:00.051570 16051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:00.072846 16051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:00.074324 16051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:00.074460 16051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:00.074968 16051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:00.075058 16051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:00.075125 16051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:00.075153 16051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:15:00 dist-test-slave-7vjv krb5kdc[15391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317300, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
I20260809 23:15:00.082326 16051 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:45921
I20260809 23:15:00.082405 16280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:45921 every 8 connection(s)
I20260809 23:15:00.082665 16051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:15:00.084611 15587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.075305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:58629 (local address 127.2.76.254:38423)
0809 23:15:00.075349 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:00.075352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:00.075622 (+ 270us) server_negotiation.cc:407] Connection header received
0809 23:15:00.076244 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:00.076245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:00.076257 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:00.076300 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:00.077094 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.077798 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.079376 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.079694 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.079769 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.082185 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:00.082186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:00.082186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:15:00.082192 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:00.083559 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:00.083872 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:00.083873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:00.083873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:00.083919 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:00.084146 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:00.084147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:00.084147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:00.084249 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:00.084296 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:00.084482 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 23:15:00.084555 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:15:00.084810 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.075312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:38423 (local address 127.2.76.195:58629)
0809 23:15:00.075459 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:00.075502 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:00.076133 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:00.076378 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:00.076382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:00.076743 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:00.076997 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.077003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.077897 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.077899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.079287 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.079290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.079801 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.079803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.079945 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.080457 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:00.080465 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:00.082070 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:15:00.083623 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:00.083627 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:00.083629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:00.083810 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:00.083969 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:00.083970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:00.083970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:00.084060 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:00.084300 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:15:00.084305 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:15:00.084397 (+ 92us) client_negotiation.cc:769] Sending connection context
0809 23:15:00.084462 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 23:15:00.084597 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260809 23:15:00.085060 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 16051
I20260809 23:15:00.085176 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:15:00.085280 16281 heartbeater.cc:347] Connected to a master server at 127.2.76.254:38423
I20260809 23:15:00.085368 16281 heartbeater.cc:464] Registering TS with master...
I20260809 23:15:00.085368 2355 external_mini_cluster.cc:1428] Setting key 5d3c1c3ba9943ebf678274435e4c56e6
I20260809 23:15:00.085558 16281 heartbeater.cc:511] Master 127.2.76.254:38423 requested a full tablet report, sending...
I20260809 23:15:00.086131 15463 ts_manager.cc:194] Registered new tserver with Master: 45b084e5df8b4f5aa9b271e5fd63a74c (127.2.76.195:45921)
I20260809 23:15:00.086541 15463 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:58629
I20260809 23:15:00.090791 15587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.086315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:51169 (local address 127.2.76.254:38423)
0809 23:15:00.086348 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:00.086349 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:00.086458 (+ 109us) server_negotiation.cc:407] Connection header received
0809 23:15:00.086520 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:00.086521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:00.086530 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:00.086584 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:00.086895 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.087260 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.087928 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.088125 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.088183 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.088905 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:00.088906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:00.088906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:15:00.088911 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:00.090155 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:00.090392 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:00.090393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:00.090393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:00.090429 (+ 36us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:00.090524 (+ 95us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:00.090524 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:00.090524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:00.090591 (+ 67us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:00.090628 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:00.090698 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 23:15:00.090743 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260809 23:15:00.090803 16288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.086278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:38423 (local address 127.2.76.195:51169)
0809 23:15:00.086391 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:00.086399 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:00.086470 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:00.086590 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:00.086590 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:00.086769 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:00.086828 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.086830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.087322 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.087323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.087863 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.087865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.088184 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.088185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.088230 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.088551 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:00.088554 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:00.088814 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:15:00.090217 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:00.090218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:00.090218 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:00.090345 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:00.090460 (+ 115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:00.090460 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:00.090460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:00.090495 (+ 35us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:00.090621 (+ 126us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:15:00.090621 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:15:00.090648 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 23:15:00.090687 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 23:15:00.090742 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
I20260809 23:15:00.096836 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 23:15:00 dist-test-slave-7vjv krb5kdc[15391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317300, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 23:15:00 dist-test-slave-7vjv krb5kdc[15391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:00 dist-test-slave-7vjv krb5kdc[15391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 23:15:00.116796 15587 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:00.111958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36522 (local address 127.2.76.254:38423)
0809 23:15:00.112011 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:00.112012 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:00.112034 (+ 22us) server_negotiation.cc:407] Connection header received
0809 23:15:00.112145 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:00.112146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:00.112155 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:00.112217 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:00.112818 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.113252 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.113914 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.114140 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.114205 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.116705 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36522: BlockingRecv error: recv got EOF from 127.0.0.1:36522 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 09 23:15:00 dist-test-slave-7vjv krb5kdc[15391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317300, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
I20260809 23:15:00.122604 15587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.117486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36526 (local address 127.2.76.254:38423)
0809 23:15:00.117517 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:00.117519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:00.117576 (+ 57us) server_negotiation.cc:407] Connection header received
0809 23:15:00.117619 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:00.117620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:00.117626 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:00.117668 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:00.118043 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.118392 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.118943 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.119082 (+ 139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.119123 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.120899 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:00.120900 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:00.120900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:15:00.120906 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:00.121882 (+ 976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:00.122146 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:00.122147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:00.122147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:00.122190 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:00.122320 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:00.122320 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:00.122320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:00.122387 (+ 67us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:00.122442 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:00.122512 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 23:15:00.122560 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260809 23:15:00.123865 15463 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:15:00.125283 15463 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:15:00.130962 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.127184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:39419 (local address 127.0.0.1:52224)
0809 23:15:00.127521 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:00.127530 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:00.127569 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:00.127901 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:00.127903 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:00.127905 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:15:00.127947 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.127950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.129130 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.129131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.129825 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.129827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.130709 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.130710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.130778 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.130791 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 23:15:00.130851 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 23:15:00.130896 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":51,"threads_started":1}
I20260809 23:15:00.131003 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.127384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52224 (local address 127.2.76.194:39419)
0809 23:15:00.127745 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:00.127749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:00.127757 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:15:00.127790 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:00.127792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:00.127806 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:00.128036 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:15:00.128114 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.129045 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.129913 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.130600 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.130695 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.130761 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:00.130863 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 23:15:00.130899 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":100,"threads_started":1}
I20260809 23:15:00.131165 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.127183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:39391 (local address 127.0.0.1:50490)
0809 23:15:00.127319 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:00.127353 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:00.127412 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:00.127779 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:00.127781 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:00.127783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:15:00.127850 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.127851 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.129071 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.129072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.130010 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.130012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.130946 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.130947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.131009 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.131013 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:15:00.131067 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 23:15:00.131104 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"spinlock_wait_cycles":768,"thread_start_us":50,"threads_started":1}
I20260809 23:15:00.131481 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.127376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50490 (local address 127.2.76.193:39391)
0809 23:15:00.127559 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:00.127563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:00.127574 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:15:00.127614 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:00.127616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:00.127631 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:00.127701 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:15:00.127916 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.128989 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.130265 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.130778 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.130842 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.130916 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:00.131325 (+ 409us) server_negotiation.cc:299] Negotiation successful
0809 23:15:00.131386 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260809 23:15:00.131616 16321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.127557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56108 (local address 127.2.76.195:45921)
0809 23:15:00.127692 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:00.127697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:00.127706 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:15:00.127738 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:00.127739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:00.127754 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:00.127833 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:15:00.127988 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.128799 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.130140 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.130815 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.130871 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.130926 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:00.131486 (+ 560us) server_negotiation.cc:299] Negotiation successful
0809 23:15:00.131520 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260809 23:15:00.131819 16311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.127189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:45921 (local address 127.0.0.1:56108)
0809 23:15:00.127570 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:00.127580 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:00.127631 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:00.127843 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:00.127844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:00.127845 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:15:00.127883 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.127884 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.128878 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.128879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.130079 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.130081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.130962 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.130963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.131391 (+ 428us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.131396 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:15:00.131455 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 23:15:00.131504 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":41,"threads_started":1}
I20260809 23:15:00.131783 15913 tablet_service.cc:1467] Processing CreateTablet for tablet d5fd7fa7054e409892c02d5bb42c6815 (DEFAULT_TABLE table=test-table [id=41319f97908c4a13ae73ef0fce13e4ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:15:00.131994 15698 tablet_service.cc:1467] Processing CreateTablet for tablet d5fd7fa7054e409892c02d5bb42c6815 (DEFAULT_TABLE table=test-table [id=41319f97908c4a13ae73ef0fce13e4ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:15:00.132182 15913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fd7fa7054e409892c02d5bb42c6815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:00.132342 15698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fd7fa7054e409892c02d5bb42c6815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:00.132464 16167 tablet_service.cc:1467] Processing CreateTablet for tablet d5fd7fa7054e409892c02d5bb42c6815 (DEFAULT_TABLE table=test-table [id=41319f97908c4a13ae73ef0fce13e4ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:15:00.132766 16167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fd7fa7054e409892c02d5bb42c6815. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:00.135414 16326 tablet_bootstrap.cc:492] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c: Bootstrap starting.
I20260809 23:15:00.135819 16327 tablet_bootstrap.cc:492] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1: Bootstrap starting.
I20260809 23:15:00.136214 16326 tablet_bootstrap.cc:654] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:00.136475 16327 tablet_bootstrap.cc:654] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:00.136550 16326 log.cc:824] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:15:00.136822 16327 log.cc:824] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1: Log is configured to *not* fsync() on all Append() calls
I20260809 23:15:00.137295 16326 tablet_bootstrap.cc:492] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c: No bootstrap required, opened a new log
I20260809 23:15:00.137367 16326 ts_tablet_manager.cc:1397] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:15:00.137598 16327 tablet_bootstrap.cc:492] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1: No bootstrap required, opened a new log
I20260809 23:15:00.137682 16327 ts_tablet_manager.cc:1397] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:15:00.138742 16331 tablet_bootstrap.cc:492] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d: Bootstrap starting.
I20260809 23:15:00.139340 16331 tablet_bootstrap.cc:654] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:00.139367 16326 raft_consensus.cc:348] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.139528 16326 raft_consensus.cc:374] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:00.139587 16326 raft_consensus.cc:729] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45b084e5df8b4f5aa9b271e5fd63a74c, State: Initialized, Role: FOLLOWER
I20260809 23:15:00.139599 16327 raft_consensus.cc:348] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.139703 16331 log.cc:824] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:15:00.139736 16327 raft_consensus.cc:374] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:00.139698 16326 consensus_queue.cc:260] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.139775 16327 raft_consensus.cc:729] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da180222b3ef4a44885a2eb142988da1, State: Initialized, Role: FOLLOWER
I20260809 23:15:00.139878 16327 consensus_queue.cc:260] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.139951 16326 ts_tablet_manager.cc:1428] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 23:15:00.140085 16281 heartbeater.cc:503] Master 127.2.76.254:38423 was elected leader, sending a full tablet report...
I20260809 23:15:00.140161 16327 ts_tablet_manager.cc:1428] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 23:15:00.140282 15796 heartbeater.cc:503] Master 127.2.76.254:38423 was elected leader, sending a full tablet report...
I20260809 23:15:00.140755 16331 tablet_bootstrap.cc:492] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d: No bootstrap required, opened a new log
I20260809 23:15:00.140861 16331 ts_tablet_manager.cc:1397] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:15:00.142208 16331 raft_consensus.cc:348] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.142328 16331 raft_consensus.cc:374] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:00.142366 16331 raft_consensus.cc:729] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b28c3135e8d4472b58af402abdfb79d, State: Initialized, Role: FOLLOWER
I20260809 23:15:00.142500 16331 consensus_queue.cc:260] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.142764 16331 ts_tablet_manager.cc:1428] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:15:00.142810 16036 heartbeater.cc:503] Master 127.2.76.254:38423 was elected leader, sending a full tablet report...
W20260809 23:15:00.180599 16040 tablet.cc:2369] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:15:00.297271 15798 tablet.cc:2369] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:15:00.333427 16282 tablet.cc:2369] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:15:00.476583 16340 raft_consensus.cc:482] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:15:00.476702 16340 raft_consensus.cc:504] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.477124 16340 leader_election.cc:288] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da180222b3ef4a44885a2eb142988da1 (127.2.76.193:39391), 45b084e5df8b4f5aa9b271e5fd63a74c (127.2.76.195:45921)
I20260809 23:15:00.479898 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.477352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:42483 (local address 127.2.76.193:39391)
0809 23:15:00.477402 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:00.477404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:00.477519 (+ 115us) server_negotiation.cc:407] Connection header received
0809 23:15:00.477592 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:00.477593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:00.477607 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:00.477665 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:15:00.477816 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.478344 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.479228 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.479538 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.479612 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.479638 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:00.479789 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 23:15:00.479828 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 23:15:00.479907 16344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.477303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:39391 (local address 127.2.76.194:42483)
0809 23:15:00.477431 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:00.477443 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:00.477533 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:00.477685 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:00.477685 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:00.477687 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:15:00.477741 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.477745 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.478447 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.478448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.479144 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.479146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.479644 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.479645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.479710 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.479714 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:15:00.479765 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 23:15:00.479815 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260809 23:15:00.479939 16341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.477312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:45921 (local address 127.2.76.194:40667)
0809 23:15:00.477434 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:00.477443 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:00.477530 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:00.477704 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:00.477705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:00.477707 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:15:00.477792 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.477794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.478396 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.478397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.479143 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.479145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.479669 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.479670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.479739 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.479743 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:15:00.479811 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 23:15:00.479847 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
I20260809 23:15:00.479938 16321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.477393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:40667 (local address 127.2.76.195:45921)
0809 23:15:00.477438 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:00.477440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:00.477541 (+ 101us) server_negotiation.cc:407] Connection header received
0809 23:15:00.477579 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:00.477580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:00.477593 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:00.477671 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:15:00.477885 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.478321 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.479230 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.479543 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.479657 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.479664 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:00.479834 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 23:15:00.479874 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:15:00.480274 16209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fd7fa7054e409892c02d5bb42c6815" candidate_uuid: "9b28c3135e8d4472b58af402abdfb79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" is_pre_election: true
I20260809 23:15:00.480432 16209 raft_consensus.cc:2454] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b28c3135e8d4472b58af402abdfb79d in term 0.
I20260809 23:15:00.480479 15732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fd7fa7054e409892c02d5bb42c6815" candidate_uuid: "9b28c3135e8d4472b58af402abdfb79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da180222b3ef4a44885a2eb142988da1" is_pre_election: true
I20260809 23:15:00.480664 15732 raft_consensus.cc:2454] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b28c3135e8d4472b58af402abdfb79d in term 0.
I20260809 23:15:00.480873 15835 leader_election.cc:302] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b28c3135e8d4472b58af402abdfb79d, da180222b3ef4a44885a2eb142988da1; no voters:
I20260809 23:15:00.481032 16340 raft_consensus.cc:2781] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:15:00.481086 16340 raft_consensus.cc:482] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:15:00.481103 16340 raft_consensus.cc:3037] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:00.481714 16340 raft_consensus.cc:504] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.481841 16340 leader_election.cc:288] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [CANDIDATE]: Term 1 election: Requested vote from peers da180222b3ef4a44885a2eb142988da1 (127.2.76.193:39391), 45b084e5df8b4f5aa9b271e5fd63a74c (127.2.76.195:45921)
I20260809 23:15:00.481971 16209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fd7fa7054e409892c02d5bb42c6815" candidate_uuid: "9b28c3135e8d4472b58af402abdfb79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c"
I20260809 23:15:00.481971 15732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fd7fa7054e409892c02d5bb42c6815" candidate_uuid: "9b28c3135e8d4472b58af402abdfb79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da180222b3ef4a44885a2eb142988da1"
I20260809 23:15:00.482056 16209 raft_consensus.cc:3037] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:00.482075 15732 raft_consensus.cc:3037] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:00.482833 15732 raft_consensus.cc:2454] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b28c3135e8d4472b58af402abdfb79d in term 1.
I20260809 23:15:00.482833 16209 raft_consensus.cc:2454] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b28c3135e8d4472b58af402abdfb79d in term 1.
I20260809 23:15:00.483047 15839 leader_election.cc:302] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45b084e5df8b4f5aa9b271e5fd63a74c, 9b28c3135e8d4472b58af402abdfb79d; no voters:
I20260809 23:15:00.483158 16340 raft_consensus.cc:2781] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:15:00.483270 16340 raft_consensus.cc:686] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [term 1 LEADER]: Becoming Leader. State: Replica: 9b28c3135e8d4472b58af402abdfb79d, State: Running, Role: LEADER
I20260809 23:15:00.483354 16340 consensus_queue.cc:237] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.484156 15463 catalog_manager.cc:5495] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b28c3135e8d4472b58af402abdfb79d (127.2.76.194). New cstate: current_term: 1 leader_uuid: "9b28c3135e8d4472b58af402abdfb79d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } health_report { overall_health: UNKNOWN } } }
I20260809 23:15:00.573400 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.571448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52230 (local address 127.2.76.194:39419)
0809 23:15:00.571507 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:00.571509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:00.571516 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:15:00.571550 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:00.571551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:00.571568 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:00.571628 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:15:00.571768 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.572235 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.572853 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.573008 (+ 155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.573051 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.573171 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:15:00.573255 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:15:00.573305 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:00.573335 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 23:15:00.573351 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 23:15:00.575057 15732 raft_consensus.cc:1261] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Refusing update from remote peer 9b28c3135e8d4472b58af402abdfb79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:15:00.575073 16209 raft_consensus.cc:1261] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Refusing update from remote peer 9b28c3135e8d4472b58af402abdfb79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:15:00.575377 16345 consensus_queue.cc:1035] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:15:00.575515 16340 consensus_queue.cc:1035] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:15:00.577668 16351 mvcc.cc:205] Tried to move back new op lower bound from 7316755663153098752 to 7316755662780420096. Current Snapshot: MvccSnapshot[applied={T|T < 7316755663153098752}]
I20260809 23:15:00.577917 16356 mvcc.cc:205] Tried to move back new op lower bound from 7316755663153098752 to 7316755662780420096. Current Snapshot: MvccSnapshot[applied={T|T < 7316755663153098752}]
I20260809 23:15:00.578281 16355 mvcc.cc:205] Tried to move back new op lower bound from 7316755663153098752 to 7316755662780420096. Current Snapshot: MvccSnapshot[applied={T|T < 7316755663153098752 or (T in {7316755663153098752})}]
I20260809 23:15:00.579968 15465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:15:00.580113 15465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:15:00.580865 15465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0: Sending DeleteTablet for 3 replicas of tablet d5fd7fa7054e409892c02d5bb42c6815
I20260809 23:15:00.581192 15913 tablet_service.cc:1514] Processing DeleteTablet for tablet d5fd7fa7054e409892c02d5bb42c6815 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:15:00 UTC) from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:52224
I20260809 23:15:00.581208 15698 tablet_service.cc:1514] Processing DeleteTablet for tablet d5fd7fa7054e409892c02d5bb42c6815 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:15:00 UTC) from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:50490
I20260809 23:15:00.581398 16167 tablet_service.cc:1514] Processing DeleteTablet for tablet d5fd7fa7054e409892c02d5bb42c6815 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:15:00 UTC) from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:56108
I20260809 23:15:00.581532 16370 tablet_replica.cc:330] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d: stopping tablet replica
I20260809 23:15:00.581615 16367 tablet_replica.cc:330] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1: stopping tablet replica
I20260809 23:15:00.581634 16370 raft_consensus.cc:2229] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:15:00.581697 16367 raft_consensus.cc:2229] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:15:00.581615 15465 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 23:15:00.581720 16370 raft_consensus.cc:2258] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 23:15:00.581756 15465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:15:00.581776 16367 raft_consensus.cc:2258] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:15:00.582149 16371 tablet_replica.cc:330] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c: stopping tablet replica
I20260809 23:15:00.582191 16370 ts_tablet_manager.cc:1905] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:15:00.582265 16371 raft_consensus.cc:2229] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:15:00.582409 16371 raft_consensus.cc:2258] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:15:00.583204 15698 tablet_service.cc:1467] Processing CreateTablet for tablet 9079af66cd1b43d0a08bb1be2fd5f393 (DEFAULT_TABLE table=test-table [id=83168655e3b24cf09b884ac253342cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:15:00.583230 15913 tablet_service.cc:1467] Processing CreateTablet for tablet 9079af66cd1b43d0a08bb1be2fd5f393 (DEFAULT_TABLE table=test-table [id=83168655e3b24cf09b884ac253342cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:15:00.583321 15913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9079af66cd1b43d0a08bb1be2fd5f393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:00.583315 15698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9079af66cd1b43d0a08bb1be2fd5f393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:00.583879 16370 ts_tablet_manager.cc:1918] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:15:00.583945 16370 log.cc:1196] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/d5fd7fa7054e409892c02d5bb42c6815
I20260809 23:15:00.584227 16370 ts_tablet_manager.cc:1939] T d5fd7fa7054e409892c02d5bb42c6815 P 9b28c3135e8d4472b58af402abdfb79d: Deleting consensus metadata
I20260809 23:15:00.584465 16167 tablet_service.cc:1467] Processing CreateTablet for tablet 9079af66cd1b43d0a08bb1be2fd5f393 (DEFAULT_TABLE table=test-table [id=83168655e3b24cf09b884ac253342cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:15:00.584585 16167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9079af66cd1b43d0a08bb1be2fd5f393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:00.585394 16327 tablet_bootstrap.cc:492] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1: Bootstrap starting.
I20260809 23:15:00.585829 16327 tablet_bootstrap.cc:654] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:00.586063 16367 ts_tablet_manager.cc:1905] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:15:00.586251 16326 tablet_bootstrap.cc:492] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c: Bootstrap starting.
I20260809 23:15:00.586618 16326 tablet_bootstrap.cc:654] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:00.586941 15443 catalog_manager.cc:4841] TS 9b28c3135e8d4472b58af402abdfb79d (127.2.76.194:39419): tablet d5fd7fa7054e409892c02d5bb42c6815 (table test-table [id=41319f97908c4a13ae73ef0fce13e4ac]) successfully deleted
I20260809 23:15:00.587131 16331 tablet_bootstrap.cc:492] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d: Bootstrap starting.
I20260809 23:15:00.587280 16326 tablet_bootstrap.cc:492] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c: No bootstrap required, opened a new log
I20260809 23:15:00.587329 16326 ts_tablet_manager.cc:1397] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 23:15:00.587534 16331 tablet_bootstrap.cc:654] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:00.587509 16326 raft_consensus.cc:348] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.587576 16326 raft_consensus.cc:374] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:00.587605 16326 raft_consensus.cc:729] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45b084e5df8b4f5aa9b271e5fd63a74c, State: Initialized, Role: FOLLOWER
I20260809 23:15:00.587673 16326 consensus_queue.cc:260] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.587766 16326 ts_tablet_manager.cc:1428] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 23:15:00.587976 16367 ts_tablet_manager.cc:1918] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:15:00.588083 16367 log.cc:1196] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/wal/wals/d5fd7fa7054e409892c02d5bb42c6815
I20260809 23:15:00.588148 16371 ts_tablet_manager.cc:1905] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:15:00.588459 16367 ts_tablet_manager.cc:1939] T d5fd7fa7054e409892c02d5bb42c6815 P da180222b3ef4a44885a2eb142988da1: Deleting consensus metadata
I20260809 23:15:00.588918 15441 catalog_manager.cc:4841] TS da180222b3ef4a44885a2eb142988da1 (127.2.76.193:39391): tablet d5fd7fa7054e409892c02d5bb42c6815 (table test-table [id=41319f97908c4a13ae73ef0fce13e4ac]) successfully deleted
I20260809 23:15:00.589113 16327 tablet_bootstrap.cc:492] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1: No bootstrap required, opened a new log
I20260809 23:15:00.589154 16327 ts_tablet_manager.cc:1397] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 23:15:00.589298 16327 raft_consensus.cc:348] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.589375 16327 raft_consensus.cc:374] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:00.589416 16327 raft_consensus.cc:729] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da180222b3ef4a44885a2eb142988da1, State: Initialized, Role: FOLLOWER
I20260809 23:15:00.589442 16331 tablet_bootstrap.cc:492] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d: No bootstrap required, opened a new log
I20260809 23:15:00.589457 16327 consensus_queue.cc:260] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.589488 16331 ts_tablet_manager.cc:1397] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:15:00.589653 16331 raft_consensus.cc:348] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.589721 16331 raft_consensus.cc:374] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:00.589751 16331 raft_consensus.cc:729] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b28c3135e8d4472b58af402abdfb79d, State: Initialized, Role: FOLLOWER
I20260809 23:15:00.589771 16327 ts_tablet_manager.cc:1428] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:15:00.589787 16331 consensus_queue.cc:260] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.589871 16331 ts_tablet_manager.cc:1428] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 23:15:00.590360 16371 ts_tablet_manager.cc:1918] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:15:00.590451 16371 log.cc:1196] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/d5fd7fa7054e409892c02d5bb42c6815
I20260809 23:15:00.590705 16371 ts_tablet_manager.cc:1939] T d5fd7fa7054e409892c02d5bb42c6815 P 45b084e5df8b4f5aa9b271e5fd63a74c: Deleting consensus metadata
I20260809 23:15:00.591048 15449 catalog_manager.cc:4841] TS 45b084e5df8b4f5aa9b271e5fd63a74c (127.2.76.195:45921): tablet d5fd7fa7054e409892c02d5bb42c6815 (table test-table [id=41319f97908c4a13ae73ef0fce13e4ac]) successfully deleted
I20260809 23:15:00.737933 16334 raft_consensus.cc:482] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:15:00.738026 16334 raft_consensus.cc:504] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.738374 16334 leader_election.cc:288] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da180222b3ef4a44885a2eb142988da1 (127.2.76.193:39391), 9b28c3135e8d4472b58af402abdfb79d (127.2.76.194:39419)
I20260809 23:15:00.741333 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.738611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:46503 (local address 127.2.76.194:39419)
0809 23:15:00.738655 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:00.738657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:00.738790 (+ 133us) server_negotiation.cc:407] Connection header received
0809 23:15:00.738828 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:00.738833 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:00.738840 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:00.738903 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:15:00.739083 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.739480 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.740466 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.740982 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.741054 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.741060 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:00.741236 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 23:15:00.741276 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:15:00.741339 16379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.738571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:39419 (local address 127.2.76.195:46503)
0809 23:15:00.738684 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:00.738693 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:00.738782 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:00.738918 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:00.738919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:00.738921 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:15:00.738987 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.738989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.739566 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.739567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.740373 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.740376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.741058 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.741060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.741149 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.741155 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 23:15:00.741213 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 23:15:00.741258 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260809 23:15:00.741356 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.738628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:43943 (local address 127.2.76.193:39391)
0809 23:15:00.738674 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:00.738676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:00.738762 (+ 86us) server_negotiation.cc:407] Connection header received
0809 23:15:00.738813 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:00.738814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:00.738824 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:00.738872 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:15:00.739033 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.739481 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.740546 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.741004 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.741044 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.741052 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:00.741266 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 23:15:00.741300 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:15:00.741566 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.738538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:39391 (local address 127.2.76.195:43943)
0809 23:15:00.738673 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:00.738683 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:00.738742 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:00.738895 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:00.738896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:00.738898 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:15:00.738958 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.738967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.739590 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.739591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.740421 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.740424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.741126 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.741127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.741177 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.741179 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:15:00.741431 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 23:15:00.741469 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":12,"thread_start_us":60,"threads_started":1}
I20260809 23:15:00.741817 15732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9079af66cd1b43d0a08bb1be2fd5f393" candidate_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da180222b3ef4a44885a2eb142988da1" is_pre_election: true
I20260809 23:15:00.741811 15955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9079af66cd1b43d0a08bb1be2fd5f393" candidate_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b28c3135e8d4472b58af402abdfb79d" is_pre_election: true
I20260809 23:15:00.741914 15732 raft_consensus.cc:2454] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45b084e5df8b4f5aa9b271e5fd63a74c in term 0.
I20260809 23:15:00.741928 15955 raft_consensus.cc:2454] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45b084e5df8b4f5aa9b271e5fd63a74c in term 0.
I20260809 23:15:00.742084 16086 leader_election.cc:302] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45b084e5df8b4f5aa9b271e5fd63a74c, da180222b3ef4a44885a2eb142988da1; no voters:
I20260809 23:15:00.742220 16334 raft_consensus.cc:2781] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:15:00.742288 16334 raft_consensus.cc:482] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:15:00.742316 16334 raft_consensus.cc:3037] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:00.742957 16334 raft_consensus.cc:504] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.743100 16334 leader_election.cc:288] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [CANDIDATE]: Term 1 election: Requested vote from peers da180222b3ef4a44885a2eb142988da1 (127.2.76.193:39391), 9b28c3135e8d4472b58af402abdfb79d (127.2.76.194:39419)
I20260809 23:15:00.743225 15732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9079af66cd1b43d0a08bb1be2fd5f393" candidate_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da180222b3ef4a44885a2eb142988da1"
I20260809 23:15:00.743237 15955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9079af66cd1b43d0a08bb1be2fd5f393" candidate_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b28c3135e8d4472b58af402abdfb79d"
I20260809 23:15:00.743299 15955 raft_consensus.cc:3037] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:00.743310 15732 raft_consensus.cc:3037] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:00.743852 15955 raft_consensus.cc:2454] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45b084e5df8b4f5aa9b271e5fd63a74c in term 1.
I20260809 23:15:00.743911 15732 raft_consensus.cc:2454] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45b084e5df8b4f5aa9b271e5fd63a74c in term 1.
I20260809 23:15:00.744026 16088 leader_election.cc:302] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45b084e5df8b4f5aa9b271e5fd63a74c, 9b28c3135e8d4472b58af402abdfb79d; no voters:
I20260809 23:15:00.744143 16334 raft_consensus.cc:2781] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:15:00.744254 16334 raft_consensus.cc:686] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 LEADER]: Becoming Leader. State: Replica: 45b084e5df8b4f5aa9b271e5fd63a74c, State: Running, Role: LEADER
I20260809 23:15:00.744314 16334 consensus_queue.cc:237] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.744875 15463 catalog_manager.cc:5495] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c reported cstate change: term changed from 0 to 1, leader changed from <none> to 45b084e5df8b4f5aa9b271e5fd63a74c (127.2.76.195). New cstate: current_term: 1 leader_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } health_report { overall_health: HEALTHY } } }
Aug 09 23:15:00 dist-test-slave-7vjv krb5kdc[15391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317299, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.254@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
I20260809 23:15:00.790462 16394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.784724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36540 (local address 127.2.76.254:38423)
0809 23:15:00.784823 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:00.784824 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:00.784883 (+ 59us) server_negotiation.cc:407] Connection header received
0809 23:15:00.784915 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:00.784916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:00.784942 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:00.785042 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:00.785474 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.785872 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.786507 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.786683 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.786727 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.788488 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:00.788489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:00.788489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:15:00.788495 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:00.789557 (+ 1062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:00.789874 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:00.789874 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:00.789875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:00.789923 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:00.790096 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:00.790097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:00.790097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:00.790188 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:00.790236 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:00.790338 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 23:15:00.790403 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":55,"threads_started":1}
I20260809 23:15:00.790608 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:00.784685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:38423 (local address 127.0.0.1:36540)
0809 23:15:00.784817 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:00.784828 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:00.784881 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:00.785077 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:00.785078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:00.785379 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:00.785415 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.785416 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.785932 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.785932 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.786439 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:00.786441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:00.786739 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:00.786740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:00.786776 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:00.787191 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:00.787195 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:00.788420 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:15:00.789621 (+ 1201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:00.789627 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:00.789629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:00.789802 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:00.789991 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:00.789992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:00.789992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:00.790047 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:00.790238 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:15:00.790244 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:15:00.790279 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 23:15:00.790325 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 23:15:00.790451 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":49,"thread_start_us":84,"threads_started":1}
I20260809 23:15:00.792264 15463 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:36540:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 23:15:00.792425 15463 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:15:00.793642 15698 tablet_service.cc:1467] Processing CreateTablet for tablet 2129862b1b164db59d07e4703330396d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=afdaf330d11c4cc7a57163ff1ac650a8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:15:00.793663 15913 tablet_service.cc:1467] Processing CreateTablet for tablet 2129862b1b164db59d07e4703330396d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=afdaf330d11c4cc7a57163ff1ac650a8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:15:00.793709 16167 tablet_service.cc:1467] Processing CreateTablet for tablet 2129862b1b164db59d07e4703330396d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=afdaf330d11c4cc7a57163ff1ac650a8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:15:00.793772 15698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2129862b1b164db59d07e4703330396d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:00.793799 15913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2129862b1b164db59d07e4703330396d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:00.793911 16167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2129862b1b164db59d07e4703330396d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:00.794884 16327 tablet_bootstrap.cc:492] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1: Bootstrap starting.
I20260809 23:15:00.795230 16331 tablet_bootstrap.cc:492] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d: Bootstrap starting.
I20260809 23:15:00.795238 16326 tablet_bootstrap.cc:492] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c: Bootstrap starting.
I20260809 23:15:00.795286 16327 tablet_bootstrap.cc:654] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:00.795568 16331 tablet_bootstrap.cc:654] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:00.795645 16326 tablet_bootstrap.cc:654] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:00.795989 16327 tablet_bootstrap.cc:492] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1: No bootstrap required, opened a new log
I20260809 23:15:00.796051 16327 ts_tablet_manager.cc:1397] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:15:00.796209 16327 raft_consensus.cc:348] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.796283 16327 raft_consensus.cc:374] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:00.796305 16327 raft_consensus.cc:729] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da180222b3ef4a44885a2eb142988da1, State: Initialized, Role: FOLLOWER
I20260809 23:15:00.796312 16331 tablet_bootstrap.cc:492] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d: No bootstrap required, opened a new log
I20260809 23:15:00.796342 16327 consensus_queue.cc:260] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.796360 16331 ts_tablet_manager.cc:1397] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:15:00.796424 16327 ts_tablet_manager.cc:1428] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:00.796460 16331 raft_consensus.cc:348] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.796521 16331 raft_consensus.cc:374] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:00.796545 16331 raft_consensus.cc:729] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b28c3135e8d4472b58af402abdfb79d, State: Initialized, Role: FOLLOWER
I20260809 23:15:00.796590 16331 consensus_queue.cc:260] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.796677 16331 ts_tablet_manager.cc:1428] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:00.796788 16345 tablet_replica.cc:439] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } } }
I20260809 23:15:00.796799 16337 tablet_replica.cc:439] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } } }
I20260809 23:15:00.796865 16345 tablet_replica.cc:442] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:15:00.796892 16337 tablet_replica.cc:442] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:15:00.798242 16326 tablet_bootstrap.cc:492] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c: No bootstrap required, opened a new log
I20260809 23:15:00.798313 16326 ts_tablet_manager.cc:1397] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 23:15:00.798446 16326 raft_consensus.cc:348] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.798514 16326 raft_consensus.cc:374] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:00.798542 16326 raft_consensus.cc:729] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45b084e5df8b4f5aa9b271e5fd63a74c, State: Initialized, Role: FOLLOWER
I20260809 23:15:00.798588 16326 consensus_queue.cc:260] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:00.798673 16326 ts_tablet_manager.cc:1428] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:00.798708 16334 tablet_replica.cc:439] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } } }
I20260809 23:15:00.798790 16334 tablet_replica.cc:442] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:15:01.107345 16334 raft_consensus.cc:482] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:15:01.107417 16334 raft_consensus.cc:504] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.107569 16334 leader_election.cc:288] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da180222b3ef4a44885a2eb142988da1 (127.2.76.193:39391), 9b28c3135e8d4472b58af402abdfb79d (127.2.76.194:39419)
I20260809 23:15:01.107795 15955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2129862b1b164db59d07e4703330396d" candidate_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b28c3135e8d4472b58af402abdfb79d" is_pre_election: true
I20260809 23:15:01.107806 15732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2129862b1b164db59d07e4703330396d" candidate_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da180222b3ef4a44885a2eb142988da1" is_pre_election: true
I20260809 23:15:01.107901 15732 raft_consensus.cc:2454] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45b084e5df8b4f5aa9b271e5fd63a74c in term 0.
I20260809 23:15:01.107906 15955 raft_consensus.cc:2454] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45b084e5df8b4f5aa9b271e5fd63a74c in term 0.
I20260809 23:15:01.108060 16086 leader_election.cc:302] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45b084e5df8b4f5aa9b271e5fd63a74c, da180222b3ef4a44885a2eb142988da1; no voters:
I20260809 23:15:01.108206 16334 raft_consensus.cc:2781] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:15:01.108253 16334 raft_consensus.cc:482] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:15:01.108275 16334 raft_consensus.cc:3037] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:01.108953 16334 raft_consensus.cc:504] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.109102 16334 leader_election.cc:288] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [CANDIDATE]: Term 1 election: Requested vote from peers da180222b3ef4a44885a2eb142988da1 (127.2.76.193:39391), 9b28c3135e8d4472b58af402abdfb79d (127.2.76.194:39419)
I20260809 23:15:01.109297 15955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2129862b1b164db59d07e4703330396d" candidate_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b28c3135e8d4472b58af402abdfb79d"
I20260809 23:15:01.109292 15732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2129862b1b164db59d07e4703330396d" candidate_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da180222b3ef4a44885a2eb142988da1"
I20260809 23:15:01.109369 15955 raft_consensus.cc:3037] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:01.109369 15732 raft_consensus.cc:3037] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:01.110131 15955 raft_consensus.cc:2454] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45b084e5df8b4f5aa9b271e5fd63a74c in term 1.
I20260809 23:15:01.110131 15732 raft_consensus.cc:2454] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45b084e5df8b4f5aa9b271e5fd63a74c in term 1.
I20260809 23:15:01.110294 16088 leader_election.cc:302] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45b084e5df8b4f5aa9b271e5fd63a74c, 9b28c3135e8d4472b58af402abdfb79d; no voters:
I20260809 23:15:01.110500 16334 raft_consensus.cc:2781] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:15:01.110558 16334 raft_consensus.cc:686] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 LEADER]: Becoming Leader. State: Replica: 45b084e5df8b4f5aa9b271e5fd63a74c, State: Running, Role: LEADER
I20260809 23:15:01.110611 16334 consensus_queue.cc:237] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.110841 16382 tablet_replica.cc:439] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c: TxnStatusTablet state changed. Reason: New leader 45b084e5df8b4f5aa9b271e5fd63a74c. Latest consensus state: current_term: 1 leader_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } } }
I20260809 23:15:01.110929 16382 tablet_replica.cc:442] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:15:01.111059 16405 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 23:15:01.111161 16405 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 23:15:01.111155 15463 catalog_manager.cc:5495] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c reported cstate change: term changed from 0 to 1, leader changed from <none> to 45b084e5df8b4f5aa9b271e5fd63a74c (127.2.76.195). New cstate: current_term: 1 leader_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } health_report { overall_health: HEALTHY } } }
I20260809 23:15:01.154048 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:01.151138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:45921 (local address 127.0.0.1:56124)
0809 23:15:01.151223 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:01.151234 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:01.151275 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:01.151469 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:01.151470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:01.151471 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:15:01.151524 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:01.151527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.152210 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.152211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:01.153148 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:01.153151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.153552 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.153553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:01.153610 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:01.153628 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:15:01.153858 (+ 230us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:15:01.153865 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 23:15:01.153917 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 23:15:01.153966 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 23:15:01.154053 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:01.151190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56124 (local address 127.2.76.195:45921)
0809 23:15:01.151312 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:01.151314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:01.151322 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:15:01.151361 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:01.151362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:01.151377 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:01.151439 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:15:01.151591 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.152105 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.153228 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.153487 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.153532 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:01.153710 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:15:01.153795 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:15:01.153846 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:01.153938 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 23:15:01.153960 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":66,"threads_started":1}
I20260809 23:15:01.155418 15732 raft_consensus.cc:1261] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Refusing update from remote peer 45b084e5df8b4f5aa9b271e5fd63a74c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:15:01.155462 15955 raft_consensus.cc:1261] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Refusing update from remote peer 45b084e5df8b4f5aa9b271e5fd63a74c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:15:01.155797 16382 consensus_queue.cc:1035] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:15:01.155949 16334 consensus_queue.cc:1035] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c [LEADER]: Connected to new peer: Peer: permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:15:01.156527 16337 tablet_replica.cc:439] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1: TxnStatusTablet state changed. Reason: New leader 45b084e5df8b4f5aa9b271e5fd63a74c. Latest consensus state: current_term: 1 leader_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } } }
I20260809 23:15:01.156617 16337 tablet_replica.cc:442] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:15:01.156628 16345 tablet_replica.cc:439] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d: TxnStatusTablet state changed. Reason: New leader 45b084e5df8b4f5aa9b271e5fd63a74c. Latest consensus state: current_term: 1 leader_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } } }
I20260809 23:15:01.156718 16345 tablet_replica.cc:442] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:15:01.157526 16382 tablet_replica.cc:439] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } } }
I20260809 23:15:01.157596 16382 tablet_replica.cc:442] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:15:01.157769 16382 tablet_replica.cc:439] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } } }
I20260809 23:15:01.157835 16382 tablet_replica.cc:442] T 2129862b1b164db59d07e4703330396d P 45b084e5df8b4f5aa9b271e5fd63a74c: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:15:01.158206 16345 tablet_replica.cc:439] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } } }
I20260809 23:15:01.158288 16345 tablet_replica.cc:442] T 2129862b1b164db59d07e4703330396d P 9b28c3135e8d4472b58af402abdfb79d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:15:01.158692 16337 tablet_replica.cc:439] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } } }
I20260809 23:15:01.158792 16337 tablet_replica.cc:442] T 2129862b1b164db59d07e4703330396d P da180222b3ef4a44885a2eb142988da1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:15:01.162268 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:01.160248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56136 (local address 127.2.76.195:45921)
0809 23:15:01.160281 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:01.160283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:01.160397 (+ 114us) server_negotiation.cc:407] Connection header received
0809 23:15:01.160459 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:01.160460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:01.160468 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:01.160513 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:15:01.160617 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.161008 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.161661 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.161837 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.161921 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:01.162061 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:15:01.162088 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:15:01.162135 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:01.162190 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 23:15:01.162207 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:15:01.166702 16409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:01.164273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:43099 (local address 127.2.76.195:45921)
0809 23:15:01.164305 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:01.164306 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:01.164476 (+ 170us) server_negotiation.cc:407] Connection header received
0809 23:15:01.164523 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:01.164523 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:01.164529 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:01.164568 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:15:01.164692 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.165096 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.166090 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.166282 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.166334 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:01.166510 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:15:01.166529 (+ 19us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:15:01.166562 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:01.166617 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 23:15:01.166648 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":15}
I20260809 23:15:01.166769 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:01.164245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:45921 (local address 127.2.76.195:43099)
0809 23:15:01.164402 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:01.164413 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:01.164473 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:01.164572 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:01.164573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:01.164574 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:15:01.164610 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:01.164611 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.165166 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.165167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:01.165991 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:01.166002 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.166364 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.166365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:01.166416 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:01.166431 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:15:01.166567 (+ 136us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:15:01.166569 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:15:01.166614 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 23:15:01.166665 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":2,"thread_start_us":75,"threads_started":1}
I20260809 23:15:01.168757 15955 raft_consensus.cc:1261] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Refusing update from remote peer 45b084e5df8b4f5aa9b271e5fd63a74c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:15:01.168807 15732 raft_consensus.cc:1261] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Refusing update from remote peer 45b084e5df8b4f5aa9b271e5fd63a74c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:15:01.168990 16334 consensus_queue.cc:1035] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:15:01.169145 16432 consensus_queue.cc:1035] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [LEADER]: Connected to new peer: Peer: permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:15:01.180007 15467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:15:01.180120 15467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:15:01.180790 15467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0: Sending DeleteTablet for 3 replicas of tablet 9079af66cd1b43d0a08bb1be2fd5f393
I20260809 23:15:01.181034 15913 tablet_service.cc:1514] Processing DeleteTablet for tablet 9079af66cd1b43d0a08bb1be2fd5f393 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:15:01 UTC) from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:52224
I20260809 23:15:01.181061 15698 tablet_service.cc:1514] Processing DeleteTablet for tablet 9079af66cd1b43d0a08bb1be2fd5f393 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:15:01 UTC) from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:50490
I20260809 23:15:01.181252 16167 tablet_service.cc:1514] Processing DeleteTablet for tablet 9079af66cd1b43d0a08bb1be2fd5f393 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:15:01 UTC) from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:56108
I20260809 23:15:01.181260 16438 tablet_replica.cc:330] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1: stopping tablet replica
I20260809 23:15:01.181317 16438 raft_consensus.cc:2229] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:15:01.181348 16438 raft_consensus.cc:2258] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:15:01.181438 16437 tablet_replica.cc:330] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d: stopping tablet replica
I20260809 23:15:01.181501 16437 raft_consensus.cc:2229] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:15:01.181530 16437 raft_consensus.cc:2258] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:15:01.181753 16437 ts_tablet_manager.cc:1905] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:15:01.181789 16438 ts_tablet_manager.cc:1905] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:15:01.183624 16437 ts_tablet_manager.cc:1918] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:15:01.183691 16437 log.cc:1196] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/9079af66cd1b43d0a08bb1be2fd5f393
I20260809 23:15:01.183642 16438 ts_tablet_manager.cc:1918] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:15:01.183844 16438 log.cc:1196] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/wal/wals/9079af66cd1b43d0a08bb1be2fd5f393
I20260809 23:15:01.183905 16437 ts_tablet_manager.cc:1939] T 9079af66cd1b43d0a08bb1be2fd5f393 P 9b28c3135e8d4472b58af402abdfb79d: Deleting consensus metadata
I20260809 23:15:01.184176 16438 ts_tablet_manager.cc:1939] T 9079af66cd1b43d0a08bb1be2fd5f393 P da180222b3ef4a44885a2eb142988da1: Deleting consensus metadata
I20260809 23:15:01.184226 15443 catalog_manager.cc:4841] TS 9b28c3135e8d4472b58af402abdfb79d (127.2.76.194:39419): tablet 9079af66cd1b43d0a08bb1be2fd5f393 (table test-table [id=83168655e3b24cf09b884ac253342cf7]) successfully deleted
I20260809 23:15:01.184608 15441 catalog_manager.cc:4841] TS da180222b3ef4a44885a2eb142988da1 (127.2.76.193:39391): tablet 9079af66cd1b43d0a08bb1be2fd5f393 (table test-table [id=83168655e3b24cf09b884ac253342cf7]) successfully deleted
I20260809 23:15:01.188321 16439 tablet_replica.cc:330] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c: stopping tablet replica
I20260809 23:15:01.188392 16439 raft_consensus.cc:2229] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:15:01.188470 16439 raft_consensus.cc:2258] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:15:01.189766 16439 ts_tablet_manager.cc:1905] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:15:01.191386 16439 ts_tablet_manager.cc:1918] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:15:01.191453 16439 log.cc:1196] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/9079af66cd1b43d0a08bb1be2fd5f393
I20260809 23:15:01.191686 16439 ts_tablet_manager.cc:1939] T 9079af66cd1b43d0a08bb1be2fd5f393 P 45b084e5df8b4f5aa9b271e5fd63a74c: Deleting consensus metadata
I20260809 23:15:01.192023 15449 catalog_manager.cc:4841] TS 45b084e5df8b4f5aa9b271e5fd63a74c (127.2.76.195:45921): tablet 9079af66cd1b43d0a08bb1be2fd5f393 (table test-table [id=83168655e3b24cf09b884ac253342cf7]) successfully deleted
Aug 09 23:15:01 dist-test-slave-7vjv krb5kdc[15391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 23:15:01 dist-test-slave-7vjv krb5kdc[15391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:01 dist-test-slave-7vjv krb5kdc[15391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 23:15:01.212706 16394 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:01.203036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36546 (local address 127.2.76.254:38423)
0809 23:15:01.203111 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:01.203125 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:01.203308 (+ 183us) server_negotiation.cc:407] Connection header received
0809 23:15:01.203337 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:01.203338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:01.203350 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:01.203403 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:01.203950 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.204775 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.205489 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.205801 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.205879 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:01.212646 (+ 6767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36546: BlockingRecv error: recv got EOF from 127.0.0.1:36546 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 23:15:01 dist-test-slave-7vjv krb5kdc[15391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM
I20260809 23:15:01.220064 16394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:01.213474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36556 (local address 127.2.76.254:38423)
0809 23:15:01.213536 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:01.213538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:01.213582 (+ 44us) server_negotiation.cc:407] Connection header received
0809 23:15:01.213638 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:01.213639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:01.213648 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:01.213696 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:01.214125 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.214566 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.215225 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.215418 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.215487 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:01.217626 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:01.217627 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:01.217628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:15:01.217635 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:01.219042 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:01.219335 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:01.219336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:01.219336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:01.219390 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:01.219577 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:01.219578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:01.219578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:01.219699 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:01.219794 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:01.219888 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 23:15:01.219964 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 23:15:01.221671 15467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:15:01.221843 15467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:15:01.223240 15698 tablet_service.cc:1467] Processing CreateTablet for tablet 9dd372e36dda479d907583b7170d30f9 (DEFAULT_TABLE table=test-table [id=7a91fd4853144141b81e7320359bb556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:15:01.223240 15913 tablet_service.cc:1467] Processing CreateTablet for tablet 9dd372e36dda479d907583b7170d30f9 (DEFAULT_TABLE table=test-table [id=7a91fd4853144141b81e7320359bb556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:15:01.223366 15913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dd372e36dda479d907583b7170d30f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:01.223366 15698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dd372e36dda479d907583b7170d30f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:01.223357 16167 tablet_service.cc:1467] Processing CreateTablet for tablet 9dd372e36dda479d907583b7170d30f9 (DEFAULT_TABLE table=test-table [id=7a91fd4853144141b81e7320359bb556]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:15:01.223475 16167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dd372e36dda479d907583b7170d30f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:01.224709 16327 tablet_bootstrap.cc:492] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1: Bootstrap starting.
I20260809 23:15:01.224822 16331 tablet_bootstrap.cc:492] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d: Bootstrap starting.
I20260809 23:15:01.225049 16327 tablet_bootstrap.cc:654] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:01.225164 16331 tablet_bootstrap.cc:654] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:01.225374 16326 tablet_bootstrap.cc:492] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c: Bootstrap starting.
I20260809 23:15:01.225654 16327 tablet_bootstrap.cc:492] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1: No bootstrap required, opened a new log
I20260809 23:15:01.225711 16327 ts_tablet_manager.cc:1397] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:15:01.225747 16326 tablet_bootstrap.cc:654] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:01.225860 16327 raft_consensus.cc:348] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.225925 16327 raft_consensus.cc:374] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:01.225946 16327 raft_consensus.cc:729] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da180222b3ef4a44885a2eb142988da1, State: Initialized, Role: FOLLOWER
I20260809 23:15:01.225987 16327 consensus_queue.cc:260] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.226073 16327 ts_tablet_manager.cc:1428] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 23:15:01.226495 16331 tablet_bootstrap.cc:492] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d: No bootstrap required, opened a new log
I20260809 23:15:01.226543 16331 ts_tablet_manager.cc:1397] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:15:01.226642 16331 raft_consensus.cc:348] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.226709 16331 raft_consensus.cc:374] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:01.226728 16331 raft_consensus.cc:729] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b28c3135e8d4472b58af402abdfb79d, State: Initialized, Role: FOLLOWER
I20260809 23:15:01.226768 16331 consensus_queue.cc:260] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.226855 16331 ts_tablet_manager.cc:1428] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:01.227072 16326 tablet_bootstrap.cc:492] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c: No bootstrap required, opened a new log
I20260809 23:15:01.227123 16326 ts_tablet_manager.cc:1397] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:15:01.227259 16326 raft_consensus.cc:348] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.227329 16326 raft_consensus.cc:374] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:01.227347 16326 raft_consensus.cc:729] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45b084e5df8b4f5aa9b271e5fd63a74c, State: Initialized, Role: FOLLOWER
I20260809 23:15:01.227385 16326 consensus_queue.cc:260] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.227486 16326 ts_tablet_manager.cc:1428] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:01.326216 16345 raft_consensus.cc:482] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:15:01.326342 16345 raft_consensus.cc:504] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.326543 16345 leader_election.cc:288] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da180222b3ef4a44885a2eb142988da1 (127.2.76.193:39391), 45b084e5df8b4f5aa9b271e5fd63a74c (127.2.76.195:45921)
I20260809 23:15:01.326691 15732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dd372e36dda479d907583b7170d30f9" candidate_uuid: "9b28c3135e8d4472b58af402abdfb79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da180222b3ef4a44885a2eb142988da1" is_pre_election: true
I20260809 23:15:01.326777 15732 raft_consensus.cc:2454] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b28c3135e8d4472b58af402abdfb79d in term 0.
I20260809 23:15:01.326741 16209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dd372e36dda479d907583b7170d30f9" candidate_uuid: "9b28c3135e8d4472b58af402abdfb79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" is_pre_election: true
I20260809 23:15:01.326848 16209 raft_consensus.cc:2454] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b28c3135e8d4472b58af402abdfb79d in term 0.
I20260809 23:15:01.326933 15835 leader_election.cc:302] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b28c3135e8d4472b58af402abdfb79d, da180222b3ef4a44885a2eb142988da1; no voters:
I20260809 23:15:01.327003 16345 raft_consensus.cc:2781] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:15:01.327028 16345 raft_consensus.cc:482] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:15:01.327052 16345 raft_consensus.cc:3037] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:01.327745 16345 raft_consensus.cc:504] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.327881 16345 leader_election.cc:288] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [CANDIDATE]: Term 1 election: Requested vote from peers da180222b3ef4a44885a2eb142988da1 (127.2.76.193:39391), 45b084e5df8b4f5aa9b271e5fd63a74c (127.2.76.195:45921)
I20260809 23:15:01.328055 16209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dd372e36dda479d907583b7170d30f9" candidate_uuid: "9b28c3135e8d4472b58af402abdfb79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c"
I20260809 23:15:01.328049 15732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dd372e36dda479d907583b7170d30f9" candidate_uuid: "9b28c3135e8d4472b58af402abdfb79d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da180222b3ef4a44885a2eb142988da1"
I20260809 23:15:01.328163 16209 raft_consensus.cc:3037] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:01.328163 15732 raft_consensus.cc:3037] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:01.328723 15732 raft_consensus.cc:2454] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b28c3135e8d4472b58af402abdfb79d in term 1.
I20260809 23:15:01.328873 15835 leader_election.cc:302] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b28c3135e8d4472b58af402abdfb79d, da180222b3ef4a44885a2eb142988da1; no voters:
I20260809 23:15:01.328945 16209 raft_consensus.cc:2454] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b28c3135e8d4472b58af402abdfb79d in term 1.
I20260809 23:15:01.328972 16345 raft_consensus.cc:2781] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:15:01.329068 16345 raft_consensus.cc:686] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [term 1 LEADER]: Becoming Leader. State: Replica: 9b28c3135e8d4472b58af402abdfb79d, State: Running, Role: LEADER
I20260809 23:15:01.329123 16345 consensus_queue.cc:237] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.329715 15465 catalog_manager.cc:5495] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b28c3135e8d4472b58af402abdfb79d (127.2.76.194). New cstate: current_term: 1 leader_uuid: "9b28c3135e8d4472b58af402abdfb79d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } health_report { overall_health: UNKNOWN } } }
I20260809 23:15:01.342757 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:01.340759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52238 (local address 127.2.76.194:39419)
0809 23:15:01.340900 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:01.340902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:01.340907 (+ 5us) server_negotiation.cc:407] Connection header received
0809 23:15:01.340924 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:01.340924 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:01.340933 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:01.340980 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:15:01.341129 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.341558 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.342215 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.342378 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.342429 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:01.342553 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:15:01.342577 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:15:01.342612 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:01.342653 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 23:15:01.342667 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260809 23:15:01.343376 16209 raft_consensus.cc:1261] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Refusing update from remote peer 9b28c3135e8d4472b58af402abdfb79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:15:01.343410 15732 raft_consensus.cc:1261] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Refusing update from remote peer 9b28c3135e8d4472b58af402abdfb79d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:15:01.343573 16463 consensus_queue.cc:1035] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:15:01.343664 16345 consensus_queue.cc:1035] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [LEADER]: Connected to new peer: Peer: permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:15:01.346266 15467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:15:01.346351 15467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:15:01.346885 15467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0: Sending DeleteTablet for 3 replicas of tablet 9dd372e36dda479d907583b7170d30f9
I20260809 23:15:01.347105 15913 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dd372e36dda479d907583b7170d30f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:15:01 UTC) from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:52224
I20260809 23:15:01.347136 15698 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dd372e36dda479d907583b7170d30f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:15:01 UTC) from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:50490
I20260809 23:15:01.347201 16438 tablet_replica.cc:330] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1: stopping tablet replica
I20260809 23:15:01.347200 16437 tablet_replica.cc:330] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d: stopping tablet replica
I20260809 23:15:01.347183 16167 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dd372e36dda479d907583b7170d30f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:15:01 UTC) from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:56108
I20260809 23:15:01.347234 16438 raft_consensus.cc:2229] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:15:01.347257 16437 raft_consensus.cc:2229] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:15:01.347292 16438 raft_consensus.cc:2258] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:15:01.347311 16437 raft_consensus.cc:2258] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:15:01.347390 16439 tablet_replica.cc:330] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c: stopping tablet replica
I20260809 23:15:01.347441 16439 raft_consensus.cc:2229] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:15:01.347486 16439 raft_consensus.cc:2258] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:15:01.347590 16438 ts_tablet_manager.cc:1905] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:15:01.347620 15467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:15:01.347748 15467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:15:01.347740 16439 ts_tablet_manager.cc:1905] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:15:01.349047 16438 ts_tablet_manager.cc:1918] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:15:01.349112 16438 log.cc:1196] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-0/wal/wals/9dd372e36dda479d907583b7170d30f9
I20260809 23:15:01.349123 16439 ts_tablet_manager.cc:1918] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:15:01.349174 16439 log.cc:1196] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/9dd372e36dda479d907583b7170d30f9
I20260809 23:15:01.349360 16438 ts_tablet_manager.cc:1939] T 9dd372e36dda479d907583b7170d30f9 P da180222b3ef4a44885a2eb142988da1: Deleting consensus metadata
I20260809 23:15:01.349433 16439 ts_tablet_manager.cc:1939] T 9dd372e36dda479d907583b7170d30f9 P 45b084e5df8b4f5aa9b271e5fd63a74c: Deleting consensus metadata
I20260809 23:15:01.349691 15441 catalog_manager.cc:4841] TS da180222b3ef4a44885a2eb142988da1 (127.2.76.193:39391): tablet 9dd372e36dda479d907583b7170d30f9 (table test-table [id=7a91fd4853144141b81e7320359bb556]) successfully deleted
I20260809 23:15:01.349731 15449 catalog_manager.cc:4841] TS 45b084e5df8b4f5aa9b271e5fd63a74c (127.2.76.195:45921): tablet 9dd372e36dda479d907583b7170d30f9 (table test-table [id=7a91fd4853144141b81e7320359bb556]) successfully deleted
I20260809 23:15:01.349730 15698 tablet_service.cc:1467] Processing CreateTablet for tablet 75f8a251b3864442b997a9ea464f636c (DEFAULT_TABLE table=test-table [id=6a25e78c4dac42e5a343206676d3ce96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:15:01.349828 15698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75f8a251b3864442b997a9ea464f636c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:01.349851 16167 tablet_service.cc:1467] Processing CreateTablet for tablet 75f8a251b3864442b997a9ea464f636c (DEFAULT_TABLE table=test-table [id=6a25e78c4dac42e5a343206676d3ce96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:15:01.349953 16167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75f8a251b3864442b997a9ea464f636c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:01.349967 15913 tablet_service.cc:1467] Processing CreateTablet for tablet 75f8a251b3864442b997a9ea464f636c (DEFAULT_TABLE table=test-table [id=6a25e78c4dac42e5a343206676d3ce96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:15:01.350066 15913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75f8a251b3864442b997a9ea464f636c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:01.350657 16327 tablet_bootstrap.cc:492] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1: Bootstrap starting.
I20260809 23:15:01.350991 16327 tablet_bootstrap.cc:654] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:01.351027 16326 tablet_bootstrap.cc:492] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c: Bootstrap starting.
I20260809 23:15:01.351352 16326 tablet_bootstrap.cc:654] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:01.351356 16331 tablet_bootstrap.cc:492] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d: Bootstrap starting.
I20260809 23:15:01.351653 16331 tablet_bootstrap.cc:654] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:01.351794 16437 ts_tablet_manager.cc:1905] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:15:01.351914 16327 tablet_bootstrap.cc:492] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1: No bootstrap required, opened a new log
I20260809 23:15:01.351970 16327 ts_tablet_manager.cc:1397] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:15:01.352150 16327 raft_consensus.cc:348] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.352197 16331 tablet_bootstrap.cc:492] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d: No bootstrap required, opened a new log
I20260809 23:15:01.352217 16327 raft_consensus.cc:374] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:01.352231 16327 raft_consensus.cc:729] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da180222b3ef4a44885a2eb142988da1, State: Initialized, Role: FOLLOWER
I20260809 23:15:01.352236 16331 ts_tablet_manager.cc:1397] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:15:01.352280 16327 consensus_queue.cc:260] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.352325 16331 raft_consensus.cc:348] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.352363 16327 ts_tablet_manager.cc:1428] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:01.352380 16331 raft_consensus.cc:374] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:01.352397 16331 raft_consensus.cc:729] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b28c3135e8d4472b58af402abdfb79d, State: Initialized, Role: FOLLOWER
I20260809 23:15:01.352442 16331 consensus_queue.cc:260] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.352546 16331 ts_tablet_manager.cc:1428] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:01.353195 16437 ts_tablet_manager.cc:1918] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:15:01.353255 16437 log.cc:1196] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/9dd372e36dda479d907583b7170d30f9
I20260809 23:15:01.353457 16437 ts_tablet_manager.cc:1939] T 9dd372e36dda479d907583b7170d30f9 P 9b28c3135e8d4472b58af402abdfb79d: Deleting consensus metadata
I20260809 23:15:01.353741 15443 catalog_manager.cc:4841] TS 9b28c3135e8d4472b58af402abdfb79d (127.2.76.194:39419): tablet 9dd372e36dda479d907583b7170d30f9 (table test-table [id=7a91fd4853144141b81e7320359bb556]) successfully deleted
I20260809 23:15:01.354086 16326 tablet_bootstrap.cc:492] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c: No bootstrap required, opened a new log
I20260809 23:15:01.354146 16326 ts_tablet_manager.cc:1397] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 23:15:01.354233 16326 raft_consensus.cc:348] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.354275 16326 raft_consensus.cc:374] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:01.354286 16326 raft_consensus.cc:729] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45b084e5df8b4f5aa9b271e5fd63a74c, State: Initialized, Role: FOLLOWER
I20260809 23:15:01.354331 16326 consensus_queue.cc:260] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.354420 16326 ts_tablet_manager.cc:1428] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:01.689678 16432 raft_consensus.cc:482] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:15:01.689798 16432 raft_consensus.cc:504] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.689993 16432 leader_election.cc:288] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da180222b3ef4a44885a2eb142988da1 (127.2.76.193:39391), 9b28c3135e8d4472b58af402abdfb79d (127.2.76.194:39419)
I20260809 23:15:01.690237 15732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f8a251b3864442b997a9ea464f636c" candidate_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da180222b3ef4a44885a2eb142988da1" is_pre_election: true
I20260809 23:15:01.690237 15955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f8a251b3864442b997a9ea464f636c" candidate_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b28c3135e8d4472b58af402abdfb79d" is_pre_election: true
I20260809 23:15:01.690352 15955 raft_consensus.cc:2454] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45b084e5df8b4f5aa9b271e5fd63a74c in term 0.
I20260809 23:15:01.690352 15732 raft_consensus.cc:2454] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45b084e5df8b4f5aa9b271e5fd63a74c in term 0.
I20260809 23:15:01.690510 16088 leader_election.cc:302] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45b084e5df8b4f5aa9b271e5fd63a74c, 9b28c3135e8d4472b58af402abdfb79d; no voters:
I20260809 23:15:01.690609 16432 raft_consensus.cc:2781] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:15:01.690667 16432 raft_consensus.cc:482] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:15:01.690685 16432 raft_consensus.cc:3037] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:01.691365 16432 raft_consensus.cc:504] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.691491 16432 leader_election.cc:288] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [CANDIDATE]: Term 1 election: Requested vote from peers da180222b3ef4a44885a2eb142988da1 (127.2.76.193:39391), 9b28c3135e8d4472b58af402abdfb79d (127.2.76.194:39419)
I20260809 23:15:01.691660 15732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f8a251b3864442b997a9ea464f636c" candidate_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da180222b3ef4a44885a2eb142988da1"
I20260809 23:15:01.691655 15955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75f8a251b3864442b997a9ea464f636c" candidate_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b28c3135e8d4472b58af402abdfb79d"
I20260809 23:15:01.691751 15732 raft_consensus.cc:3037] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:01.691751 15955 raft_consensus.cc:3037] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:01.692402 15955 raft_consensus.cc:2454] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45b084e5df8b4f5aa9b271e5fd63a74c in term 1.
I20260809 23:15:01.692411 15732 raft_consensus.cc:2454] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45b084e5df8b4f5aa9b271e5fd63a74c in term 1.
I20260809 23:15:01.692571 16088 leader_election.cc:302] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45b084e5df8b4f5aa9b271e5fd63a74c, 9b28c3135e8d4472b58af402abdfb79d; no voters:
I20260809 23:15:01.692680 16432 raft_consensus.cc:2781] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:15:01.692835 16432 raft_consensus.cc:686] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 LEADER]: Becoming Leader. State: Replica: 45b084e5df8b4f5aa9b271e5fd63a74c, State: Running, Role: LEADER
I20260809 23:15:01.692899 16432 consensus_queue.cc:237] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } }
I20260809 23:15:01.693423 15467 catalog_manager.cc:5495] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c reported cstate change: term changed from 0 to 1, leader changed from <none> to 45b084e5df8b4f5aa9b271e5fd63a74c (127.2.76.195). New cstate: current_term: 1 leader_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45b084e5df8b4f5aa9b271e5fd63a74c" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45921 } health_report { overall_health: HEALTHY } } }
I20260809 23:15:01.707652 16488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:01.705794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56150 (local address 127.2.76.195:45921)
0809 23:15:01.705918 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:01.705920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:01.705927 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:15:01.705944 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:01.705945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:01.705953 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:01.706013 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:15:01.706122 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.706548 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.707155 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:01.707321 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:01.707366 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:01.707480 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:15:01.707510 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:15:01.707542 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:01.707588 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 23:15:01.707603 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260809 23:15:01.709528 15955 raft_consensus.cc:1261] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Refusing update from remote peer 45b084e5df8b4f5aa9b271e5fd63a74c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:15:01.709573 15732 raft_consensus.cc:1261] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Refusing update from remote peer 45b084e5df8b4f5aa9b271e5fd63a74c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:15:01.709756 16432 consensus_queue.cc:1035] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [LEADER]: Connected to new peer: Peer: permanent_uuid: "da180222b3ef4a44885a2eb142988da1" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:15:01.709887 16481 consensus_queue.cc:1035] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b28c3135e8d4472b58af402abdfb79d" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 39419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:15:01.719544 15467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:15:01.719663 15467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:15:01.720386 15467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b56ddb436aea4b0694eefbeca8f07ec0: Sending DeleteTablet for 3 replicas of tablet 75f8a251b3864442b997a9ea464f636c
I20260809 23:15:01.720621 15698 tablet_service.cc:1514] Processing DeleteTablet for tablet 75f8a251b3864442b997a9ea464f636c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:15:01 UTC) from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:50490
I20260809 23:15:01.720674 15913 tablet_service.cc:1514] Processing DeleteTablet for tablet 75f8a251b3864442b997a9ea464f636c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:15:01 UTC) from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:52224
I20260809 23:15:01.720723 16438 tablet_replica.cc:330] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1: stopping tablet replica
I20260809 23:15:01.720759 16437 tablet_replica.cc:330] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d: stopping tablet replica
I20260809 23:15:01.720784 16438 raft_consensus.cc:2229] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:15:01.720798 16437 raft_consensus.cc:2229] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:15:01.720825 16438 raft_consensus.cc:2258] T 75f8a251b3864442b997a9ea464f636c P da180222b3ef4a44885a2eb142988da1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:15:01.720829 16437 raft_consensus.cc:2258] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:15:01.720894 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 15590
I20260809 23:15:01.721555 16437 ts_tablet_manager.cc:1905] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:15:01.723121 16437 ts_tablet_manager.cc:1918] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:15:01.723165 16437 log.cc:1196] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/75f8a251b3864442b997a9ea464f636c
I20260809 23:15:01.723331 16437 ts_tablet_manager.cc:1939] T 75f8a251b3864442b997a9ea464f636c P 9b28c3135e8d4472b58af402abdfb79d: Deleting consensus metadata
I20260809 23:15:01.723598 15443 catalog_manager.cc:4841] TS 9b28c3135e8d4472b58af402abdfb79d (127.2.76.194:39419): tablet 75f8a251b3864442b997a9ea464f636c (table test-table [id=6a25e78c4dac42e5a343206676d3ce96]) successfully deleted
I20260809 23:15:01.729079 16167 tablet_service.cc:1514] Processing DeleteTablet for tablet 75f8a251b3864442b997a9ea464f636c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:15:01 UTC) from {username='oryx', principal='oryx/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:56108
I20260809 23:15:01.729189 16439 tablet_replica.cc:330] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c: stopping tablet replica
I20260809 23:15:01.729254 16439 raft_consensus.cc:2229] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:15:01.729311 16439 raft_consensus.cc:2258] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:15:01.730398 16439 ts_tablet_manager.cc:1905] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:15:01.732075 16439 ts_tablet_manager.cc:1918] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:15:01.732168 16439 log.cc:1196] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/75f8a251b3864442b997a9ea464f636c
I20260809 23:15:01.732411 16439 ts_tablet_manager.cc:1939] T 75f8a251b3864442b997a9ea464f636c P 45b084e5df8b4f5aa9b271e5fd63a74c: Deleting consensus metadata
I20260809 23:15:01.732713 15449 catalog_manager.cc:4841] TS 45b084e5df8b4f5aa9b271e5fd63a74c (127.2.76.195:45921): tablet 75f8a251b3864442b997a9ea464f636c (table test-table [id=6a25e78c4dac42e5a343206676d3ce96]) successfully deleted
W20260809 23:15:01.735213 15441 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:39391 (error 108)
I20260809 23:15:01.735666 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 15811
I20260809 23:15:01.735720 16496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:01.735559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:39391 (local address 127.0.0.1:50498)
0809 23:15:01.735648 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:01.735689 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":36,"threads_started":1}
W20260809 23:15:01.742801 15441 catalog_manager.cc:4584] TS da180222b3ef4a44885a2eb142988da1 (127.2.76.193:39391): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 75f8a251b3864442b997a9ea464f636c: Network error: Client connection negotiation failed: client connection to 127.2.76.193:39391: connect: Connection refused (error 111)
I20260809 23:15:01.751036 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 16051
I20260809 23:15:01.764158 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 15419
2026-08-09T23:15:01Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3542 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:15:01 dist-test-slave-7vjv krb5kdc[16501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:15:01 dist-test-slave-7vjv krb5kdc[16501](info): set up 2 sockets
Aug 09 23:15:01 dist-test-slave-7vjv krb5kdc[16501](info): commencing operation
krb5kdc: starting...
W20260809 23:15:02.933032 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:15:03 dist-test-slave-7vjv krb5kdc[16501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:15:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:15:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:15:03.036963 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:38087
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:38087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:15:03.108805 16526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:03.108959 16526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:03.108978 16526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:03.109879 16526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:15:03.109906 16526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:03.109918 16526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:03.109929 16526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:15:03.109939 16526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:15:03.111141 16526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:38087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:38087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:03.111295 16526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:03.111430 16526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:03.115517 16544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:03.115494 16542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:03.115579 16526 server_base.cc:1034] running on GCE node
W20260809 23:15:03.115521 16541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:03.115863 16526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:03.116163 16526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:03.117327 16526 hybrid_clock.cc:648] HybridClock initialized: now 1786317303117304 us; error 34 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T23:15:03Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1351 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:15:03 dist-test-slave-7vjv krb5kdc[16556](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:15:03 dist-test-slave-7vjv krb5kdc[16556](info): set up 2 sockets
Aug 09 23:15:03 dist-test-slave-7vjv krb5kdc[16556](info): commencing operation
krb5kdc: starting...
W20260809 23:15:04.274127 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:15:04 dist-test-slave-7vjv krb5kdc[16556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:15:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:15:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:15:04.386219 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:40469
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:40469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:15:04.460835 16583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:04.460995 16583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:04.461014 16583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:04.461902 16583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:15:04.461930 16583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:04.461942 16583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:04.461953 16583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:15:04.461963 16583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:15:04.463140 16583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:40469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:40469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:04.463295 16583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:04.463436 16583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:04.467826 16589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:04.467839 16591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:04.467916 16583 server_base.cc:1034] running on GCE node
W20260809 23:15:04.467831 16593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:04.468343 16583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:04.468623 16583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:04.469794 16583 hybrid_clock.cc:648] HybridClock initialized: now 1786317304469792 us; error 37 us; skew 500 ppm
Aug 09 23:15:04 dist-test-slave-7vjv krb5kdc[16556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317304, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:15:04.471895 16583 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:15:04.472326 16583 webserver.cc:460] Webserver started at http://127.2.76.254:41567/ using document root <none> and password file <none>
I20260809 23:15:04.472553 16583 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:04.472626 16583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:04.472766 16583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:04.473667 16583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "57d755c1397445008269312c2057a82c"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "c69833eba3bf4baa029e64c8dc464349"
server_key_iv: "fd9359e4598d5004068965bb396b8831"
server_key_version: "encryptionkey@0"
I20260809 23:15:04.473994 16583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "57d755c1397445008269312c2057a82c"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "c69833eba3bf4baa029e64c8dc464349"
server_key_iv: "fd9359e4598d5004068965bb396b8831"
server_key_version: "encryptionkey@0"
I20260809 23:15:04.475356 16583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:15:04.476060 16600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:04.476284 16583 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:15:04.476378 16583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "57d755c1397445008269312c2057a82c"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "c69833eba3bf4baa029e64c8dc464349"
server_key_iv: "fd9359e4598d5004068965bb396b8831"
server_key_version: "encryptionkey@0"
I20260809 23:15:04.476470 16583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:04.491472 16583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:04.493374 16583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:04.493528 16583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:04.497377 16583 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:40469
I20260809 23:15:04.497404 16696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:40469 every 8 connection(s)
I20260809 23:15:04.497702 16583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:15:04.498503 16701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:04.500409 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 16583
I20260809 23:15:04.500523 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:15:04.500711 2355 external_mini_cluster.cc:1428] Setting key ecb219c18995618028b44ee2f66c6963
I20260809 23:15:04.500700 16701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c: Bootstrap starting.
I20260809 23:15:04.501399 16701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:04.501690 16701 log.cc:824] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:15:04.502393 16701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c: No bootstrap required, opened a new log
I20260809 23:15:04.503850 16701 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d755c1397445008269312c2057a82c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40469 } }
I20260809 23:15:04.504019 16701 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:04.504053 16701 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57d755c1397445008269312c2057a82c, State: Initialized, Role: FOLLOWER
I20260809 23:15:04.504187 16701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d755c1397445008269312c2057a82c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40469 } }
I20260809 23:15:04.504263 16701 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:15:04.504295 16701 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:15:04.504335 16701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:04.504901 16701 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d755c1397445008269312c2057a82c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40469 } }
I20260809 23:15:04.504980 16701 leader_election.cc:302] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57d755c1397445008269312c2057a82c; no voters:
I20260809 23:15:04.505185 16701 leader_election.cc:288] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:15:04.505208 16710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:15:04.505316 16710 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 1 LEADER]: Becoming Leader. State: Replica: 57d755c1397445008269312c2057a82c, State: Running, Role: LEADER
I20260809 23:15:04.505414 16701 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:15:04.505442 16710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d755c1397445008269312c2057a82c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40469 } }
I20260809 23:15:04.505798 16710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57d755c1397445008269312c2057a82c. Latest consensus state: current_term: 1 leader_uuid: "57d755c1397445008269312c2057a82c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d755c1397445008269312c2057a82c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40469 } } }
I20260809 23:15:04.505930 16710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [sys.catalog]: This master's current role is: LEADER
I20260809 23:15:04.506215 16717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:15:04.506742 16717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:15:04.507263 16714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57d755c1397445008269312c2057a82c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d755c1397445008269312c2057a82c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40469 } } }
I20260809 23:15:04.507373 16714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [sys.catalog]: This master's current role is: LEADER
Aug 09 23:15:04 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:15:04.508602 16717 catalog_manager.cc:1347] Generated new cluster ID: 533d8999feec4d74b6161176bb86e9a2
I20260809 23:15:04.508656 16717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:15:04.511749 16707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:04.501251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39348 (local address 127.2.76.254:40469)
0809 23:15:04.501403 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:04.501411 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:04.501454 (+ 43us) server_negotiation.cc:407] Connection header received
0809 23:15:04.501923 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:04.501930 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:04.501983 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:04.502123 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:04.502817 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:04.503670 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:04.504794 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:04.505257 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:04.505620 (+ 363us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:04.508040 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:04.508043 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:04.508044 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:15:04.508053 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:04.509867 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:04.510227 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:04.510230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:04.510230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:04.510323 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:04.510538 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:04.510539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:04.510539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:04.510806 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:04.510942 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:04.511399 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 23:15:04.511532 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260809 23:15:04.520025 16717 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:15:04.520494 16717 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:15:04.525790 16717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c: Generated new TSK 0
I20260809 23:15:04.525945 16717 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:15:04.553138 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40469
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:15:04.625629 16742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:04.625806 16742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:04.625825 16742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:04.626804 16742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:04.626839 16742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:04.626861 16742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:15:04.628059 16742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:04.628283 16742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:04.628454 16742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:04.633183 16749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:04.633178 16752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:04.633147 16748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:04.633527 16742 server_base.cc:1034] running on GCE node
I20260809 23:15:04.633735 16742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:04.633946 16742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:04.635085 16742 hybrid_clock.cc:648] HybridClock initialized: now 1786317304635081 us; error 35 us; skew 500 ppm
Aug 09 23:15:04 dist-test-slave-7vjv krb5kdc[16556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317304, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:15:04.637657 16742 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:15:04.638048 16742 webserver.cc:460] Webserver started at http://127.2.76.193:45433/ using document root <none> and password file <none>
I20260809 23:15:04.638252 16742 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:04.638309 16742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:04.638439 16742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:04.639364 16742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "0660e74879f44ad7a4d010f4da53acbe"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "73a69401b842777e8f999ccea98d907b"
server_key_iv: "b7267585e62a11afaa2c15f1ec0b9798"
server_key_version: "encryptionkey@0"
I20260809 23:15:04.639716 16742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "0660e74879f44ad7a4d010f4da53acbe"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "73a69401b842777e8f999ccea98d907b"
server_key_iv: "b7267585e62a11afaa2c15f1ec0b9798"
server_key_version: "encryptionkey@0"
I20260809 23:15:04.641130 16742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:15:04.641906 16761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:04.642112 16742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:15:04.642204 16742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "0660e74879f44ad7a4d010f4da53acbe"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "73a69401b842777e8f999ccea98d907b"
server_key_iv: "b7267585e62a11afaa2c15f1ec0b9798"
server_key_version: "encryptionkey@0"
I20260809 23:15:04.642277 16742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:04.653510 16742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:04.655417 16742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:04.655557 16742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:04.655743 16742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:15:04.655992 16742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:04.656041 16742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:04.656122 16742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:04.656150 16742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:04.661540 16742 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:36717
I20260809 23:15:04.661608 16970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:36717 every 8 connection(s)
I20260809 23:15:04.661862 16742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:15:04 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317304, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:15:04.667117 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 16742
I20260809 23:15:04.667219 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:15:04.667389 2355 external_mini_cluster.cc:1428] Setting key 598cbe2b92685d54a5b3b6e483a7ba51
I20260809 23:15:04.669960 16707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:04.662483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:51799 (local address 127.2.76.254:40469)
0809 23:15:04.662529 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:04.662531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:04.662697 (+ 166us) server_negotiation.cc:407] Connection header received
0809 23:15:04.663069 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:04.663070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:04.663079 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:04.663124 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:04.663883 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:04.664400 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:04.665307 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:04.665472 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:04.665524 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:04.667297 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:04.667299 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:04.667299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:15:04.667305 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:04.668600 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:04.669041 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:04.669042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:04.669042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:04.669086 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:04.669333 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:04.669333 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:04.669333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:04.669414 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:04.669457 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:04.669764 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 23:15:04.669851 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 23:15:04.669996 16974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:04.662432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:51799)
0809 23:15:04.662585 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:04.662619 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:04.662988 (+ 369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:04.663171 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:04.663173 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:04.663657 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:04.663816 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:04.663820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:04.664512 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:04.664513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:04.665235 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:04.665238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:04.665540 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:04.665541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:04.665630 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:04.665977 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:04.665984 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:04.667203 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:15:04.668677 (+ 1474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:04.668685 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:04.668687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:04.668971 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:04.669143 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:04.669143 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:04.669144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:04.669276 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:04.669469 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:15:04.669475 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:15:04.669619 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 23:15:04.669725 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 23:15:04.669874 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260809 23:15:04.670419 16971 heartbeater.cc:347] Connected to a master server at 127.2.76.254:40469
I20260809 23:15:04.670521 16971 heartbeater.cc:464] Registering TS with master...
I20260809 23:15:04.670682 16971 heartbeater.cc:511] Master 127.2.76.254:40469 requested a full tablet report, sending...
I20260809 23:15:04.671204 16627 ts_manager.cc:194] Registered new tserver with Master: 0660e74879f44ad7a4d010f4da53acbe (127.2.76.193:36717)
I20260809 23:15:04.671973 16627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:51799
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:15:04.690891 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40469
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:15:04.762274 16978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:04.762452 16978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:04.762475 16978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:04.763463 16978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:04.763494 16978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:04.763523 16978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:15:04.764703 16978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:04.764855 16978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:04.764993 16978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:04.769164 16986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:04.769174 16992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:04.769176 16987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:04.769572 16978 server_base.cc:1034] running on GCE node
I20260809 23:15:04.769737 16978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:04.769893 16978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:04.771018 16978 hybrid_clock.cc:648] HybridClock initialized: now 1786317304771010 us; error 24 us; skew 500 ppm
Aug 09 23:15:04 dist-test-slave-7vjv krb5kdc[16556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317304, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:15:04.773576 16978 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:15:04.774039 16978 webserver.cc:460] Webserver started at http://127.2.76.194:36769/ using document root <none> and password file <none>
I20260809 23:15:04.774266 16978 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:04.774312 16978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:04.774493 16978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:04.775425 16978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "b19fd51f42214f3d9f01218be36d543e"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "d36c72514561a2ec6b49f1579d303047"
server_key_iv: "48f8d7974b071ec14341fe91cd2720bd"
server_key_version: "encryptionkey@0"
I20260809 23:15:04.775782 16978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "b19fd51f42214f3d9f01218be36d543e"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "d36c72514561a2ec6b49f1579d303047"
server_key_iv: "48f8d7974b071ec14341fe91cd2720bd"
server_key_version: "encryptionkey@0"
I20260809 23:15:04.777225 16978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:15:04.777925 17004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:04.778132 16978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:15:04.778218 16978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "b19fd51f42214f3d9f01218be36d543e"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "d36c72514561a2ec6b49f1579d303047"
server_key_iv: "48f8d7974b071ec14341fe91cd2720bd"
server_key_version: "encryptionkey@0"
I20260809 23:15:04.778262 16978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:04.787916 16978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:04.790062 16978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:04.790203 16978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:04.790367 16978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:15:04.790578 16978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:04.790613 16978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:04.790642 16978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:04.790653 16978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:04.796258 16978 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:37075
I20260809 23:15:04.796305 17201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:37075 every 8 connection(s)
I20260809 23:15:04.796562 16978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:15:04 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317304, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:15:04.804626 16707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:04.797219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:34069 (local address 127.2.76.254:40469)
0809 23:15:04.797260 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:04.797263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:04.797444 (+ 181us) server_negotiation.cc:407] Connection header received
0809 23:15:04.797855 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:04.797856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:04.797865 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:04.797920 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:04.798742 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:04.799231 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:04.800120 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:04.800300 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:04.800347 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:04.802053 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:04.802054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:04.802055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:15:04.802061 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:04.803279 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:04.803604 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:04.803604 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:04.803605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:04.803652 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:04.803942 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:04.803944 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:04.803944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:04.804071 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:04.804196 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:04.804455 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 23:15:04.804536 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:15:04.804863 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 16978
I20260809 23:15:04.804947 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:15:04.805083 2355 external_mini_cluster.cc:1428] Setting key f946587b6f4b88c64163db7db71a1a6d
I20260809 23:15:04.805363 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:04.797174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:34069)
0809 23:15:04.797337 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:04.797375 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:04.797777 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:04.797988 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:04.797991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:04.798518 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:04.798673 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:04.798677 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:04.799307 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:04.799308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:04.800024 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:04.800027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:04.800365 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:04.800366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:04.800449 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:04.800801 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:04.800808 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:04.801973 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:15:04.803340 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:04.803345 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:04.803346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:04.803544 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:04.803705 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:04.803706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:04.803706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:04.803856 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:04.804189 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:15:04.804196 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:15:04.804313 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 23:15:04.804416 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 23:15:04.804572 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
I20260809 23:15:04.805843 17203 heartbeater.cc:347] Connected to a master server at 127.2.76.254:40469
I20260809 23:15:04.805936 17203 heartbeater.cc:464] Registering TS with master...
I20260809 23:15:04.806110 17203 heartbeater.cc:511] Master 127.2.76.254:40469 requested a full tablet report, sending...
I20260809 23:15:04.807788 16624 ts_manager.cc:194] Registered new tserver with Master: b19fd51f42214f3d9f01218be36d543e (127.2.76.194:37075)
I20260809 23:15:04.808357 16624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:34069
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:15:04.828323 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40469
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:15:04.902462 17209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:04.902666 17209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:04.902702 17209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:04.903739 17209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:04.903810 17209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:04.903849 17209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:15:04.905241 17209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:04.905474 17209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:04.905761 17209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:04.910298 17226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:04.910342 17232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:04.910405 17228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:04.910705 17209 server_base.cc:1034] running on GCE node
I20260809 23:15:04.910907 17209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:04.911123 17209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:04.912287 17209 hybrid_clock.cc:648] HybridClock initialized: now 1786317304912277 us; error 34 us; skew 500 ppm
Aug 09 23:15:04 dist-test-slave-7vjv krb5kdc[16556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317304, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:15:04.914395 17209 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:15:04.914775 17209 webserver.cc:460] Webserver started at http://127.2.76.195:39921/ using document root <none> and password file <none>
I20260809 23:15:04.914964 17209 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:04.915004 17209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:04.915130 17209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:04.915961 17209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "5df7cb4de3f94605b8dcd41d77b566a4"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "e86bc3cd45bc47c0c54b32d20dd90951"
server_key_iv: "7d66378ef5c20645291cc192099d6111"
server_key_version: "encryptionkey@0"
I20260809 23:15:04.916309 17209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "5df7cb4de3f94605b8dcd41d77b566a4"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "e86bc3cd45bc47c0c54b32d20dd90951"
server_key_iv: "7d66378ef5c20645291cc192099d6111"
server_key_version: "encryptionkey@0"
I20260809 23:15:04.917563 17209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:15:04.918190 17243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:04.918350 17209 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:15:04.918430 17209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "5df7cb4de3f94605b8dcd41d77b566a4"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "e86bc3cd45bc47c0c54b32d20dd90951"
server_key_iv: "7d66378ef5c20645291cc192099d6111"
server_key_version: "encryptionkey@0"
I20260809 23:15:04.918491 17209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:04.925091 17209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:04.926564 17209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:04.926697 17209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:04.926877 17209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:15:04.927134 17209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:04.927191 17209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:04.927224 17209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:04.927248 17209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:04.932618 17209 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:37311
I20260809 23:15:04.932651 17450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:37311 every 8 connection(s)
I20260809 23:15:04.932972 17209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:15:04 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317304, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:15:04.940852 16707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:04.933635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:58535 (local address 127.2.76.254:40469)
0809 23:15:04.933696 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:04.933698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:04.933869 (+ 171us) server_negotiation.cc:407] Connection header received
0809 23:15:04.934318 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:04.934319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:04.934331 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:04.934384 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:04.935202 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:04.935668 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:04.936648 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:04.936835 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:04.936909 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:04.938558 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:04.938559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:04.938560 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:15:04.938567 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:04.939583 (+ 1016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:04.939963 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:04.939964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:04.939965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:04.940029 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:04.940286 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:04.940287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:04.940287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:04.940391 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:04.940461 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:04.940696 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 23:15:04.940774 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:15:04.940970 17453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:04.933568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:58535)
0809 23:15:04.933750 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:04.933790 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:04.934203 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:04.934439 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:04.934443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:04.934937 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:04.935093 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:04.935098 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:04.935756 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:04.935757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:04.936564 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:04.936566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:04.936922 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:04.936923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:04.937026 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:04.937359 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:04.937366 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:04.938465 (+ 1099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:15:04.939690 (+ 1225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:04.939696 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:04.939698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:04.939890 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:04.940113 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:04.940114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:04.940115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:04.940221 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:04.940453 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:15:04.940459 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:15:04.940577 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 23:15:04.940664 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 23:15:04.940790 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260809 23:15:04.941407 17451 heartbeater.cc:347] Connected to a master server at 127.2.76.254:40469
I20260809 23:15:04.941484 17451 heartbeater.cc:464] Registering TS with master...
I20260809 23:15:04.941598 17451 heartbeater.cc:511] Master 127.2.76.254:40469 requested a full tablet report, sending...
I20260809 23:15:04.941951 16624 ts_manager.cc:194] Registered new tserver with Master: 5df7cb4de3f94605b8dcd41d77b566a4 (127.2.76.195:37311)
I20260809 23:15:04.942237 16624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:58535
I20260809 23:15:04.942436 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 17209
I20260809 23:15:04.942543 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:15:04.942709 2355 external_mini_cluster.cc:1428] Setting key c241e9e76f966deaef6118f827f3237b
I20260809 23:15:04.943521 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.193.keytab.
Entry for principal oryx/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.193.keytab.
WARNING: no policy specified for oryx/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.194.keytab.
Entry for principal oryx/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.194.keytab.
WARNING: no policy specified for oryx/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.195.keytab.
Entry for principal oryx/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.195.keytab.
I20260809 23:15:05.006258 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 16742
I20260809 23:15:05.022610 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 16978
I20260809 23:15:05.031731 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 17209
I20260809 23:15:05.037755 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 16583
I20260809 23:15:05.045423 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:40469
--webserver_interface=127.2.76.254
--webserver_port=41567
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:40469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:15:05.117034 17466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:05.117193 17466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:05.117211 17466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:05.118181 17466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:15:05.118219 17466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:05.118232 17466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:05.118271 17466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:15:05.118288 17466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:15:05.119577 17466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:40469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:40469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=41567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:05.119798 17466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:05.119985 17466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:05.124089 17475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:05.124161 17474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:05.124111 17480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:05.124127 17466 server_base.cc:1034] running on GCE node
I20260809 23:15:05.124395 17466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:05.124610 17466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:05.125737 17466 hybrid_clock.cc:648] HybridClock initialized: now 1786317305125730 us; error 25 us; skew 500 ppm
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317305, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:15:05.127938 17466 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:15:05.128335 17466 webserver.cc:460] Webserver started at http://127.2.76.254:41567/ using document root <none> and password file <none>
I20260809 23:15:05.128502 17466 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:05.128551 17466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:05.129774 17466 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:15:05.130324 17490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:05.130486 17466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:15:05.130558 17466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "57d755c1397445008269312c2057a82c"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "c69833eba3bf4baa029e64c8dc464349"
server_key_iv: "fd9359e4598d5004068965bb396b8831"
server_key_version: "encryptionkey@0"
I20260809 23:15:05.130785 17466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:05.139545 17466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:05.141094 17466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:05.141182 17466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:05.144652 17466 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:40469
I20260809 23:15:05.144704 17577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:40469 every 8 connection(s)
I20260809 23:15:05.145072 17466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:15:05.146225 17578 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:15:05.146775 17578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c: Bootstrap starting.
I20260809 23:15:05.149523 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 17466
I20260809 23:15:05.149770 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:36717
--local_ip_for_outbound_sockets=127.2.76.193
--tserver_master_addrs=127.2.76.254:40469
--webserver_port=45433
--webserver_interface=127.2.76.193
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.76.193
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:15:05.155575 17578 log.cc:824] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:15:05.156939 17578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:15:05.157235 17578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c: Bootstrap complete.
I20260809 23:15:05.159193 17578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d755c1397445008269312c2057a82c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40469 } }
I20260809 23:15:05.159396 17578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57d755c1397445008269312c2057a82c, State: Initialized, Role: FOLLOWER
I20260809 23:15:05.159571 17578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d755c1397445008269312c2057a82c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40469 } }
I20260809 23:15:05.159644 17578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:15:05.159691 17578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:15:05.159732 17578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 1 FOLLOWER]: Advancing to term 2
I20260809 23:15:05.160689 17578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d755c1397445008269312c2057a82c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40469 } }
I20260809 23:15:05.160797 17578 leader_election.cc:302] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57d755c1397445008269312c2057a82c; no voters:
I20260809 23:15:05.160965 17578 leader_election.cc:288] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 23:15:05.161003 17586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 2 FOLLOWER]: Leader election won for term 2
I20260809 23:15:05.161160 17586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [term 2 LEADER]: Becoming Leader. State: Replica: 57d755c1397445008269312c2057a82c, State: Running, Role: LEADER
I20260809 23:15:05.161198 17578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:15:05.161311 17586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d755c1397445008269312c2057a82c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40469 } }
I20260809 23:15:05.161527 17590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "57d755c1397445008269312c2057a82c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d755c1397445008269312c2057a82c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40469 } } }
I20260809 23:15:05.161600 17590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [sys.catalog]: This master's current role is: LEADER
I20260809 23:15:05.162704 17589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57d755c1397445008269312c2057a82c. Latest consensus state: current_term: 2 leader_uuid: "57d755c1397445008269312c2057a82c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d755c1397445008269312c2057a82c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40469 } } }
I20260809 23:15:05.162751 17614 catalog_manager.cc:1259] Loaded cluster ID: 533d8999feec4d74b6161176bb86e9a2
I20260809 23:15:05.162791 17614 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c: loading cluster ID for follower catalog manager: success
I20260809 23:15:05.163784 17614 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c: acquiring CA information for follower catalog manager: success
I20260809 23:15:05.164085 17614 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:15:05.164220 17589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c [sys.catalog]: This master's current role is: LEADER
I20260809 23:15:05.164391 17600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:15:05.164518 17600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:15:05.164592 17600 catalog_manager.cc:1259] Loaded cluster ID: 533d8999feec4d74b6161176bb86e9a2
I20260809 23:15:05.164616 17600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:15:05.164798 17600 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:15:05.164870 17600 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 57d755c1397445008269312c2057a82c: Loaded TSK: 0
I20260809 23:15:05.164975 17600 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 23:15:05.223920 17583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:05.224076 17583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:05.224143 17583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:05.225169 17583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:05.225230 17583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:05.225268 17583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:15:05.226541 17583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.193.keytab
--principal=oryx/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:36717
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=45433
--tserver_master_addrs=127.2.76.254:40469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:05.226780 17583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:05.227011 17583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:05.231451 17626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:05.231428 17632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:05.231503 17583 server_base.cc:1034] running on GCE node
W20260809 23:15:05.231638 17624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:05.231849 17583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:05.232124 17583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:05.233317 17583 hybrid_clock.cc:648] HybridClock initialized: now 1786317305233296 us; error 34 us; skew 500 ppm
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317305, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:15:05.236194 17583 init.cc:377] Logged in from keytab as oryx/127.2.76.193@KRBTEST.COM (short username oryx)
I20260809 23:15:05.236621 17583 webserver.cc:460] Webserver started at http://127.2.76.193:45433/ using document root <none> and password file <none>
I20260809 23:15:05.236837 17583 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:05.236915 17583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:05.238164 17583 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:15:05.238796 17645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:05.238956 17583 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:15:05.239056 17583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "0660e74879f44ad7a4d010f4da53acbe"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "73a69401b842777e8f999ccea98d907b"
server_key_iv: "b7267585e62a11afaa2c15f1ec0b9798"
server_key_version: "encryptionkey@0"
I20260809 23:15:05.239322 17583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:05.249284 17583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:05.252278 17583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:05.252419 17583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:05.252790 17583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:15:05.253127 17583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:05.253188 17583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:05.253226 17583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:05.253252 17583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:05.259359 17583 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:36717
I20260809 23:15:05.259450 17836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:36717 every 8 connection(s)
I20260809 23:15:05.259756 17583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:15:05.264921 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 17583
I20260809 23:15:05.265234 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:37075
--local_ip_for_outbound_sockets=127.2.76.194
--tserver_master_addrs=127.2.76.254:40469
--webserver_port=36769
--webserver_interface=127.2.76.194
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.76.194
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:05.268476 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:05.260544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:50821)
0809 23:15:05.260715 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:05.260751 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:05.261330 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:05.261797 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:05.261801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:05.262414 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:05.262590 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.262594 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.263631 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.263632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.264409 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.264412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.265142 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.265144 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.265302 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.265777 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:05.265786 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:05.268277 (+ 2491us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":101,"threads_started":1}
W20260809 23:15:05.268589 17842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:05.260834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:50821 (local address 127.2.76.254:40469)
0809 23:15:05.261019 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:05.261026 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:05.261059 (+ 33us) server_negotiation.cc:407] Connection header received
0809 23:15:05.261579 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:05.261584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:05.261635 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:05.261764 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:05.262705 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.263520 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.264507 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.265050 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.265159 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.268416 (+ 3257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:50821: BlockingRecv error: recv got EOF from 127.2.76.193:50821 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
W20260809 23:15:05.268694 17837 heartbeater.cc:644] Failed to heartbeat to 127.2.76.254:40469 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.76.254:40469: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:05.273228 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:05.268974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:33213)
0809 23:15:05.269109 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:05.269119 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:05.269170 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:05.269351 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:05.269351 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:05.269480 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:05.269521 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.269522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.270124 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.270125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.270616 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.270617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.270995 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.270996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.271057 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.271452 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:05.271456 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:05.273165 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
W20260809 23:15:05.273355 17842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:05.269030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:33213 (local address 127.2.76.254:40469)
0809 23:15:05.269086 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:05.269088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:05.269156 (+ 68us) server_negotiation.cc:407] Connection header received
0809 23:15:05.269228 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:05.269228 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:05.269240 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:05.269349 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:05.269598 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.270036 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.270716 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.270894 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.270994 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.273283 (+ 2289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:33213: BlockingRecv error: recv got EOF from 127.2.76.193:33213 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:05.278414 17841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:05.273647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:44609)
0809 23:15:05.273725 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:05.273734 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:05.273780 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:05.273918 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:05.273918 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:05.274045 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:05.274072 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.274073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.274866 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.274867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.275529 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.275530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.275985 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.275985 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.276034 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.276442 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:05.276446 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:05.278355 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 23:15:05.278543 17842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:05.273692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:44609 (local address 127.2.76.254:40469)
0809 23:15:05.273735 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:05.273737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:05.273770 (+ 33us) server_negotiation.cc:407] Connection header received
0809 23:15:05.273828 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:05.273829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:05.273841 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:05.273915 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:05.274145 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.274770 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.275607 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.275901 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.275982 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.278470 (+ 2488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:44609: BlockingRecv error: recv got EOF from 127.2.76.193:44609 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 23:15:05.278632 17837 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 23:15:05.346097 17844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:05.346278 17844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:05.346311 17844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:05.347276 17844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:05.347342 17844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:05.347378 17844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:15:05.348709 17844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.194.keytab
--principal=oryx/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:37075
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=36769
--tserver_master_addrs=127.2.76.254:40469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:05.348990 17844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:05.349205 17844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:05.353500 17855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:05.353510 17862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:05.353663 17844 server_base.cc:1034] running on GCE node
W20260809 23:15:05.353575 17856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:05.353919 17844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:05.354095 17844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:05.355219 17844 hybrid_clock.cc:648] HybridClock initialized: now 1786317305355206 us; error 32 us; skew 500 ppm
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317305, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:15:05.357491 17844 init.cc:377] Logged in from keytab as oryx/127.2.76.194@KRBTEST.COM (short username oryx)
I20260809 23:15:05.357862 17844 webserver.cc:460] Webserver started at http://127.2.76.194:36769/ using document root <none> and password file <none>
I20260809 23:15:05.358079 17844 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:05.358183 17844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:05.359355 17844 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:15:05.359890 17876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:05.360044 17844 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:15:05.360162 17844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "b19fd51f42214f3d9f01218be36d543e"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "d36c72514561a2ec6b49f1579d303047"
server_key_iv: "48f8d7974b071ec14341fe91cd2720bd"
server_key_version: "encryptionkey@0"
I20260809 23:15:05.360421 17844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:05.371261 17844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:05.372895 17844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:05.373062 17844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:05.373255 17844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:15:05.373512 17844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:05.373560 17844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:05.373588 17844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:05.373611 17844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:05.378885 17844 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:37075
I20260809 23:15:05.378950 18064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:37075 every 8 connection(s)
I20260809 23:15:05.379298 17844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:05.385296 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 17844
I20260809 23:15:05.385608 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:37311
--local_ip_for_outbound_sockets=127.2.76.195
--tserver_master_addrs=127.2.76.254:40469
--webserver_port=39921
--webserver_interface=127.2.76.195
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.76.195
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:15:05.386256 17842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:05.380022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:52223 (local address 127.2.76.254:40469)
0809 23:15:05.380106 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:05.380109 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:05.380398 (+ 289us) server_negotiation.cc:407] Connection header received
0809 23:15:05.380877 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:05.380878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:05.380891 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:05.380941 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:05.381732 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.382187 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.383055 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.383223 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.383270 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.386213 (+ 2943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:52223: BlockingRecv error: recv got EOF from 127.2.76.194:52223 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 23:15:05.386341 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:05.380015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:52223)
0809 23:15:05.380266 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:05.380309 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:05.380782 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:05.380987 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:05.380990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:05.381493 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:05.381659 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.381663 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.382263 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.382264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.382975 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.382976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.383292 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.383293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.383384 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.383857 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:05.383869 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:05.386146 (+ 2277us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":158,"threads_started":1}
W20260809 23:15:05.387416 18065 heartbeater.cc:644] Failed to heartbeat to 127.2.76.254:40469 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.76.254:40469: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:05.391801 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:05.387681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:47589)
0809 23:15:05.387726 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:05.387733 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:05.387789 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:05.387912 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:05.387913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:05.388041 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:05.388080 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.388082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.388883 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.388884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.389309 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.389310 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.389776 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.389777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.389849 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.390211 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:05.390214 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:05.391718 (+ 1504us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 23:15:05.391867 17842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:05.387734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:47589 (local address 127.2.76.254:40469)
0809 23:15:05.387802 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:05.387821 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:05.387829 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:15:05.387853 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:05.387853 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:05.387863 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:05.387910 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:05.388189 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.388794 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.389383 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.389692 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.389765 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.391809 (+ 2044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:47589: BlockingRecv error: recv got EOF from 127.2.76.194:47589 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:05.396220 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:05.392039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:43469)
0809 23:15:05.392155 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:05.392165 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:05.392214 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:05.392324 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:05.392325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:05.392468 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:05.392508 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.392509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.393034 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.393035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.393907 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.393911 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.394240 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.394241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.394305 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.394632 (+ 327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:05.394637 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:05.396155 (+ 1518us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101}
W20260809 23:15:05.396260 17842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:05.392090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:43469 (local address 127.2.76.254:40469)
0809 23:15:05.392151 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:05.392157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:05.392201 (+ 44us) server_negotiation.cc:407] Connection header received
0809 23:15:05.392268 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:05.392268 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:05.392274 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:05.392317 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:05.392574 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.392941 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.393991 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.394155 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.394203 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.396207 (+ 2004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:43469: BlockingRecv error: recv got EOF from 127.2.76.194:43469 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 23:15:05.396348 18065 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 23:15:05.461946 18071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:05.462126 18071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:05.462158 18071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:05.463302 18071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:05.463366 18071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:05.463407 18071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:15:05.464713 18071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/krb5kdc/oryx_127.2.76.195.keytab
--principal=oryx/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:37311
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=39921
--tserver_master_addrs=127.2.76.254:40469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:05.464931 18071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:05.465126 18071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:05.469434 18081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:05.469415 18083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:05.469408 18080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:05.469623 18071 server_base.cc:1034] running on GCE node
I20260809 23:15:05.469800 18071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:05.470018 18071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:05.471215 18071 hybrid_clock.cc:648] HybridClock initialized: now 1786317305471200 us; error 41 us; skew 500 ppm
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317305, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:15:05.473402 18071 init.cc:377] Logged in from keytab as oryx/127.2.76.195@KRBTEST.COM (short username oryx)
I20260809 23:15:05.473723 18071 webserver.cc:460] Webserver started at http://127.2.76.195:39921/ using document root <none> and password file <none>
I20260809 23:15:05.473869 18071 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:05.473913 18071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:05.474972 18071 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:15:05.475455 18094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:05.475636 18071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:05.475718 18071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "5df7cb4de3f94605b8dcd41d77b566a4"
format_stamp: "Formatted at 2026-08-09 23:15:04 on dist-test-slave-7vjv"
server_key: "e86bc3cd45bc47c0c54b32d20dd90951"
server_key_iv: "7d66378ef5c20645291cc192099d6111"
server_key_version: "encryptionkey@0"
I20260809 23:15:05.475986 18071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:05.484695 18071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:05.486492 18071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:05.486593 18071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:05.486726 18071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:15:05.486984 18071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:05.487040 18071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:05.487076 18071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:05.487099 18071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:05.492309 18071 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:37311
I20260809 23:15:05.492378 18283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:37311 every 8 connection(s)
I20260809 23:15:05.492673 18071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 23:15:05.498760 17842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:05.493326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:52517 (local address 127.2.76.254:40469)
0809 23:15:05.493413 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:05.493415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:05.493585 (+ 170us) server_negotiation.cc:407] Connection header received
0809 23:15:05.493970 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:05.493971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:05.493981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:05.494029 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:05.494798 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.495251 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.496074 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.496308 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.496352 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.498709 (+ 2357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:52517: BlockingRecv error: recv got EOF from 127.2.76.195:52517 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 23:15:05.498802 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:05.493292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:52517)
0809 23:15:05.493465 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:05.493497 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:05.493880 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:05.494101 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:05.494104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:05.494578 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:05.494728 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.494732 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.495322 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.495323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.496003 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.496005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.496371 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.496372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.496457 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.496798 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:05.496805 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:05.498642 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
W20260809 23:15:05.498977 18284 heartbeater.cc:644] Failed to heartbeat to 127.2.76.254:40469 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.76.254:40469: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
I20260809 23:15:05.501441 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 18071
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:05.503072 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:05.499167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:43299)
0809 23:15:05.499218 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:05.499228 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:05.499270 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:05.499502 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:05.499503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:05.499617 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:05.499656 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.499657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.500237 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.500238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.500662 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.500664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.500991 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.500992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.501046 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.501346 (+ 300us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:05.501351 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:05.503007 (+ 1656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 23:15:05.503129 17842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:05.499354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:43299 (local address 127.2.76.254:40469)
0809 23:15:05.499409 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:05.499410 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:05.499415 (+ 5us) server_negotiation.cc:407] Connection header received
0809 23:15:05.499431 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:05.499431 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:05.499437 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:05.499491 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:05.499722 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.500169 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.500764 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.500930 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.500992 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.503074 (+ 2082us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:43299: BlockingRecv error: recv got EOF from 127.2.76.195:43299 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:05 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:05.507115 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:05.503268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:59117)
0809 23:15:05.503298 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:05.503306 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:05.503333 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:05.503462 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:05.503463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:05.503586 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:05.503606 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.503607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.504118 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.504119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.504623 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:05.504626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.504918 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.504919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:05.505050 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.505377 (+ 327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:05.505381 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:05.507044 (+ 1663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 23:15:05.507149 17842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:05.503326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:59117 (local address 127.2.76.254:40469)
0809 23:15:05.503361 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:05.503363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:05.503369 (+ 6us) server_negotiation.cc:407] Connection header received
0809 23:15:05.503391 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:05.503392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:05.503397 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:05.503458 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:05.503675 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.504037 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.504702 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.504851 (+ 149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.504919 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.507112 (+ 2193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:59117: BlockingRecv error: recv got EOF from 127.2.76.195:59117 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 23:15:05.507366 18284 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 23:15:05.508436 18293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:05.501821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39354 (local address 127.2.76.254:40469)
0809 23:15:05.501964 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:05.501966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:05.502017 (+ 51us) server_negotiation.cc:407] Connection header received
0809 23:15:05.502086 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:05.502087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:05.502097 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:05.502172 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:05.502803 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.503311 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.503977 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:05.504197 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:05.504261 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:05.505383 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:05.505387 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:05.505387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:15:05.505397 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:05.507128 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:05.507535 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:05.507538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:05.507538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:05.507625 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:05.507802 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:05.507803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:05.507803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:05.507990 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:05.508085 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:05.508242 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 23:15:05.508319 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
Aug 09 23:15:06 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:06 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:06.287410 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:06.278962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:56753)
0809 23:15:06.279077 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:06.279086 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:06.279133 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:06.279263 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:06.279264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:06.279410 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:06.279455 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:06.279457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:06.280037 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:06.280038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:06.280567 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:06.280568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:06.280876 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:06.280877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:06.280913 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:06.285784 (+ 4871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:06.285789 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:06.287350 (+ 1561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
W20260809 23:15:06.287520 18298 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:06.279011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:56753 (local address 127.2.76.254:40469)
0809 23:15:06.279127 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:06.279143 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:06.279152 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:15:06.279177 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:06.279178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:06.279189 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:06.279250 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:06.279529 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:06.279983 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:06.280626 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:06.280812 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:06.280867 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:06.287434 (+ 6567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:56753: BlockingRecv error: recv got EOF from 127.2.76.193:56753 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
Aug 09 23:15:06 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:06 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:06.401568 18300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:06.396852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:60581)
0809 23:15:06.397019 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:06.397032 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:06.397117 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:06.397275 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:06.397276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:06.397438 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:06.397483 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:06.397485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:06.398117 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:06.398119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:06.398659 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:06.398661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:06.399048 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:06.399050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:06.399113 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:06.399650 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:06.399655 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:06.401501 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":82,"threads_started":1}
W20260809 23:15:06.401633 18298 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:06.396942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:60581 (local address 127.2.76.254:40469)
0809 23:15:06.397032 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:06.397034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:06.397107 (+ 73us) server_negotiation.cc:407] Connection header received
0809 23:15:06.397179 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:06.397180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:06.397192 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:06.397268 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:06.397571 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:06.398030 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:06.398749 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:06.398940 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:06.399049 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:06.401590 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:60581: BlockingRecv error: recv got EOF from 127.2.76.194:60581 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 23:15:06 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:06 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:06.511651 18305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:06.507716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:47439)
0809 23:15:06.507837 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:06.507846 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:06.507896 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:06.508014 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:06.508014 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:06.508167 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:06.508212 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:06.508213 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:06.508743 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:06.508744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:06.509206 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:06.509207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:06.509507 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:06.509508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:06.509541 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:06.509914 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:06.509918 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:06.511596 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
W20260809 23:15:06.511765 18298 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:06.507783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:47439 (local address 127.2.76.254:40469)
0809 23:15:06.507856 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:06.507858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:06.507886 (+ 28us) server_negotiation.cc:407] Connection header received
0809 23:15:06.507941 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:06.507941 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:06.507952 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:06.507994 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:06.508269 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:06.508685 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:06.509272 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:06.509434 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:06.509489 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:06.511677 (+ 2188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:47439: BlockingRecv error: recv got EOF from 127.2.76.195:47439 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 23:15:07 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:07 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:07.292582 18309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:07.288003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:53037)
0809 23:15:07.288147 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:07.288159 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:07.288205 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:07.288351 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:07.288351 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:07.288524 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:07.288572 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:07.288574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:07.289255 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:07.289256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:07.289917 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:07.289920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:07.290216 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:07.290216 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:07.290257 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:07.290648 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:07.290652 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:07.292522 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
W20260809 23:15:07.292678 18310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:07.288082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:53037 (local address 127.2.76.254:40469)
0809 23:15:07.288243 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:07.288244 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:07.288251 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:15:07.288270 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:07.288270 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:07.288280 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:07.288338 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:07.288633 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:07.289171 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:07.289981 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:07.290156 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:07.290203 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:07.292609 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:53037: BlockingRecv error: recv got EOF from 127.2.76.193:53037 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":116,"threads_started":1}
Aug 09 23:15:07 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:07 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:07.406183 18311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:07.402100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:52883)
0809 23:15:07.402219 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:07.402229 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:07.402294 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:07.402413 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:07.402413 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:07.402564 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:07.402605 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:07.402607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:07.403162 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:07.403163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:07.403634 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:07.403635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:07.403921 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:07.403922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:07.403957 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:07.404470 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:07.404474 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:07.406128 (+ 1654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
W20260809 23:15:07.406273 18310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:07.402180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:52883 (local address 127.2.76.254:40469)
0809 23:15:07.402236 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:07.402237 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:07.402292 (+ 55us) server_negotiation.cc:407] Connection header received
0809 23:15:07.402341 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:07.402341 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:07.402350 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:07.402391 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:07.402656 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:07.403104 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:07.403690 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:07.403865 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:07.403908 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:07.406212 (+ 2304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:52883: BlockingRecv error: recv got EOF from 127.2.76.194:52883 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 09 23:15:07 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:07 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:07.516307 18312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:07.512025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:60199)
0809 23:15:07.512152 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:07.512168 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:07.512219 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:07.512342 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:07.512343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:07.512488 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:07.512535 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:07.512537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:07.513119 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:07.513120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:07.513582 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:07.513583 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:07.513863 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:07.513864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:07.513900 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:07.514318 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:07.514323 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:07.516251 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
W20260809 23:15:07.516397 18310 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:07.512130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:60199 (local address 127.2.76.254:40469)
0809 23:15:07.512170 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:07.512172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:07.512216 (+ 44us) server_negotiation.cc:407] Connection header received
0809 23:15:07.512271 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:07.512271 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:07.512281 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:07.512320 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:07.512607 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:07.513058 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:07.513648 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:07.513805 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:07.513849 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:07.516332 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:60199: BlockingRecv error: recv got EOF from 127.2.76.195:60199 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 23:15:08 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:08 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:08.297490 18313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:08.293067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:43591)
0809 23:15:08.293222 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:08.293232 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:08.293281 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:08.293411 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:08.293412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:08.293585 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:08.293636 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:08.293638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:08.294261 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:08.294262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:08.294752 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:08.294754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:08.295082 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:08.295082 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:08.295119 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:08.295521 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:08.295525 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:08.297427 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
W20260809 23:15:08.297632 18314 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:08.293151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:43591 (local address 127.2.76.254:40469)
0809 23:15:08.293264 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:08.293266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:08.293276 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:15:08.293325 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:08.293325 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:08.293336 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:08.293392 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:08.293701 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:08.294202 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:08.294814 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:08.295014 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:08.295065 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:08.297519 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:43591: BlockingRecv error: recv got EOF from 127.2.76.193:43591 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":53,"threads_started":1}
Aug 09 23:15:08 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:08 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:08.411023 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:08.406753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:48385)
0809 23:15:08.406893 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:08.406903 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:08.406954 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:08.407091 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:08.407092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:08.407232 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:08.407276 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:08.407278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:08.407893 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:08.407894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:08.408541 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:08.408544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:08.408842 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:08.408843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:08.408880 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:08.409270 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:08.409274 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:08.410954 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
W20260809 23:15:08.411130 18314 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:08.406840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:48385 (local address 127.2.76.254:40469)
0809 23:15:08.406893 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:08.406895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:08.406957 (+ 62us) server_negotiation.cc:407] Connection header received
0809 23:15:08.407002 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:08.407002 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:08.407012 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:08.407073 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:08.407328 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:08.407828 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:08.408605 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:08.408780 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:08.408831 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:08.411044 (+ 2213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:48385: BlockingRecv error: recv got EOF from 127.2.76.194:48385 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 09 23:15:08 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:08 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:08.520727 18319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:08.516738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:58563)
0809 23:15:08.516865 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:08.516874 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:08.516925 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:08.517029 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:08.517030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:08.517176 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:08.517223 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:08.517225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:08.517781 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:08.517782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:08.518281 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:08.518283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:08.518562 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:08.518562 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:08.518596 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:08.518953 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:08.518957 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:08.520645 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
W20260809 23:15:08.520828 18314 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:08.516799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:58563 (local address 127.2.76.254:40469)
0809 23:15:08.516862 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:08.516868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:08.516923 (+ 55us) server_negotiation.cc:407] Connection header received
0809 23:15:08.516974 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:08.516975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:08.516985 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:08.517025 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:08.517275 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:08.517717 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:08.518344 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:08.518498 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:08.518547 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:08.520741 (+ 2194us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:58563: BlockingRecv error: recv got EOF from 127.2.76.195:58563 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 23:15:09 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:09 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:09.302453 18320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:09.298096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:45173)
0809 23:15:09.298232 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:09.298243 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:09.298302 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:09.298433 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:09.298434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:09.298628 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:09.298673 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:09.298675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:09.299310 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:09.299311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:09.299809 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:09.299811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:09.300213 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:09.300214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:09.300281 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:09.300683 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:09.300687 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:09.302369 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
W20260809 23:15:09.302521 18321 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:09.298156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:45173 (local address 127.2.76.254:40469)
0809 23:15:09.298245 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:09.298247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:09.298301 (+ 54us) server_negotiation.cc:407] Connection header received
0809 23:15:09.298367 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:09.298368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:09.298379 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:09.298428 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:09.298749 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:09.299225 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:09.299889 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:09.300084 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:09.300210 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:09.302445 (+ 2235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:45173: BlockingRecv error: recv got EOF from 127.2.76.193:45173 (error 108)
Metrics: {"server-negotiator.queue_time_us":70,"thread_start_us":45,"threads_started":1}
Aug 09 23:15:09 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:09 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:09.415892 18322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:09.411430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:47669)
0809 23:15:09.411558 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:09.411568 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:09.411623 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:09.411759 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:09.411759 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:09.411931 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:09.411979 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:09.411980 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:09.412634 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:09.412634 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:09.413167 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:09.413173 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:09.413581 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:09.413582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:09.413649 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:09.414124 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:09.414129 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:09.415809 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":85,"threads_started":1}
W20260809 23:15:09.415982 18321 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:09.411604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:47669 (local address 127.2.76.254:40469)
0809 23:15:09.411646 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:09.411648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:09.411660 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:15:09.411682 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:09.411682 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:09.411691 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:09.411740 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:09.412066 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:09.412568 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:09.413259 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:09.413487 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:09.413579 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:09.415911 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:47669: BlockingRecv error: recv got EOF from 127.2.76.194:47669 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 09 23:15:09 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:09 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:09.525787 18325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:09.521284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:38609)
0809 23:15:09.521405 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:09.521416 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:09.521462 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:09.521637 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:09.521638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:09.521795 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:09.521838 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:09.521840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:09.522431 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:09.522432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:09.523101 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:09.523104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:09.523402 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:09.523403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:09.523456 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:09.523862 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:09.523866 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:09.525697 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
W20260809 23:15:09.525836 18321 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:09.521462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:38609 (local address 127.2.76.254:40469)
0809 23:15:09.521532 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:09.521533 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:09.521545 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:15:09.521564 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:09.521564 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:09.521574 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:09.521630 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:09.521914 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:09.522345 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:09.523171 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:09.523338 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:09.523403 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:09.525778 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:38609: BlockingRecv error: recv got EOF from 127.2.76.195:38609 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 23:15:10 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:10 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:10.307279 18326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:10.303096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:36087)
0809 23:15:10.303250 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:10.303260 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:10.303312 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:10.303443 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:10.303444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:10.303676 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:10.303748 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:10.303751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:10.304501 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:10.304502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:10.304957 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:10.304959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:10.305245 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:10.305245 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:10.305281 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:10.305665 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:10.305669 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:10.307210 (+ 1541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
W20260809 23:15:10.307400 18327 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:10.303185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:36087 (local address 127.2.76.254:40469)
0809 23:15:10.303314 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:10.303316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:10.303344 (+ 28us) server_negotiation.cc:407] Connection header received
0809 23:15:10.303365 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:10.303365 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:10.303376 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:10.303425 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:10.303836 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:10.304440 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:10.305026 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:10.305190 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:10.305232 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:10.307316 (+ 2084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:36087: BlockingRecv error: recv got EOF from 127.2.76.193:36087 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":55,"threads_started":1}
Aug 09 23:15:10 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:10 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:10.420377 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:10.416288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:59829)
0809 23:15:10.416423 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:10.416433 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:10.416496 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:10.416602 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:10.416602 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:10.416739 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:10.416783 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:10.416784 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:10.417320 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:10.417321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:10.417761 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:10.417762 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:10.418046 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:10.418047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:10.418085 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:10.418463 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:10.418467 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:10.420285 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
W20260809 23:15:10.420428 18327 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:10.416385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:59829 (local address 127.2.76.254:40469)
0809 23:15:10.416429 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:10.416431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:10.416482 (+ 51us) server_negotiation.cc:407] Connection header received
0809 23:15:10.416538 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:10.416539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:10.416548 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:10.416587 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:10.416838 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:10.417260 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:10.417827 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:10.417985 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:10.418032 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:10.420364 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:59829: BlockingRecv error: recv got EOF from 127.2.76.194:59829 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 23:15:10 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:10 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:10.530944 18330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:10.526504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:33003)
0809 23:15:10.526762 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:10.526774 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:10.526854 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:10.526981 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:10.526982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:10.527139 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:10.527183 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:10.527185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:10.527809 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:10.527810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:10.528365 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:10.528368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:10.528679 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:10.528680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:10.528738 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:10.529124 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:10.529129 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:10.530893 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":80,"threads_started":1}
W20260809 23:15:10.531073 18327 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:10.526604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:33003 (local address 127.2.76.254:40469)
0809 23:15:10.526766 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:10.526768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:10.526835 (+ 67us) server_negotiation.cc:407] Connection header received
0809 23:15:10.526916 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:10.526917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:10.526930 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:10.526976 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:10.527267 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:10.527726 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:10.528436 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:10.528610 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:10.528678 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:10.530974 (+ 2296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:33003: BlockingRecv error: recv got EOF from 127.2.76.195:33003 (error 108)
Metrics: {"server-negotiator.queue_time_us":142}
Aug 09 23:15:11 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:11 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:11.314123 18332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:11.307704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:38455)
0809 23:15:11.307847 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:11.307859 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:11.307915 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:11.308044 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:11.308045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:11.308210 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:11.308258 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:11.308259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:11.308875 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:11.308876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:11.309372 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:11.309373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:11.309695 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:11.309695 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:11.309731 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:11.312473 (+ 2742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:11.312477 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:11.314069 (+ 1592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
W20260809 23:15:11.314234 18333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:11.307755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:38455 (local address 127.2.76.254:40469)
0809 23:15:11.307900 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:11.307901 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:11.307909 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:15:11.307965 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:11.307965 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:11.307974 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:11.308023 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:11.308316 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:11.308809 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:11.309440 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:11.309635 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:11.309680 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:11.314157 (+ 4477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:38455: BlockingRecv error: recv got EOF from 127.2.76.193:38455 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
Aug 09 23:15:11 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:11 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:11.425096 18334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:11.420868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:38971)
0809 23:15:11.421008 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:11.421017 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:11.421067 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:11.421188 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:11.421189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:11.421337 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:11.421383 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:11.421384 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:11.421990 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:11.421991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:11.422436 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:11.422437 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:11.422740 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:11.422741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:11.422776 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:11.423279 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:11.423285 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:11.425002 (+ 1717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
W20260809 23:15:11.425148 18333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:11.420939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:38971 (local address 127.2.76.254:40469)
0809 23:15:11.421002 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:11.421004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:11.421073 (+ 69us) server_negotiation.cc:407] Connection header received
0809 23:15:11.421116 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:11.421116 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:11.421129 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:11.421172 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:11.421445 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:11.421926 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:11.422505 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:11.422676 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:11.422730 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:11.425087 (+ 2357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:38971: BlockingRecv error: recv got EOF from 127.2.76.194:38971 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 23:15:11 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:11 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:11.535918 18336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:11.531610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:32871)
0809 23:15:11.531839 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:11.531851 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:11.531913 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:11.532029 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:11.532030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:11.532189 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:11.532234 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:11.532236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:11.532943 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:11.532944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:11.533477 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:11.533479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:11.533816 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:11.533817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:11.533887 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:11.534330 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:11.534334 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:11.535838 (+ 1504us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":59,"threads_started":1}
W20260809 23:15:11.535966 18333 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:11.531791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:32871 (local address 127.2.76.254:40469)
0809 23:15:11.531850 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:11.531851 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:11.531899 (+ 48us) server_negotiation.cc:407] Connection header received
0809 23:15:11.531960 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:11.531961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:11.531971 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:11.532008 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:11.532299 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:11.532875 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:11.533551 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:11.533741 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:11.533803 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:11.535909 (+ 2106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:32871: BlockingRecv error: recv got EOF from 127.2.76.195:32871 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 23:15:12 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:12 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:12.319310 18337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:12.314705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:35587)
0809 23:15:12.314844 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:12.314855 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:12.314906 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:12.315121 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:12.315122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:12.315273 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:12.315337 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:12.315339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:12.316086 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:12.316087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:12.316636 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:12.316638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:12.316988 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:12.316989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:12.317060 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:12.317529 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:12.317533 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:12.319259 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
W20260809 23:15:12.319443 18339 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:12.314880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:35587 (local address 127.2.76.254:40469)
0809 23:15:12.315007 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:12.315008 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:12.315015 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:15:12.315035 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:12.315036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:12.315044 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:12.315103 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:12.315430 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:12.315999 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:12.316729 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:12.316918 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:12.316986 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:12.319361 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:35587: BlockingRecv error: recv got EOF from 127.2.76.193:35587 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
Aug 09 23:15:12 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:12 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:12.430011 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:12.425634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:53919)
0809 23:15:12.425786 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:12.425796 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:12.425851 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:12.425966 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:12.425967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:12.426123 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:12.426170 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:12.426172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:12.426818 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:12.426819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:12.427320 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:12.427321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:12.427658 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:12.427659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:12.427695 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:12.428194 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:12.428199 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:12.429952 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
W20260809 23:15:12.430148 18339 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:12.425689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:53919 (local address 127.2.76.254:40469)
0809 23:15:12.425756 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:12.425762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:12.425854 (+ 92us) server_negotiation.cc:407] Connection header received
0809 23:15:12.425900 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:12.425901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:12.425910 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:12.425947 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:12.426228 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:12.426748 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:12.427396 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:12.427562 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:12.427627 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:12.430066 (+ 2439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:53919: BlockingRecv error: recv got EOF from 127.2.76.194:53919 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 23:15:12 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:12 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:12.541265 18342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:12.536388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:48459)
0809 23:15:12.536522 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:12.536534 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:12.536599 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:12.536779 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:12.536781 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:12.537043 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:12.537112 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:12.537115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:12.537868 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:12.537869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:12.538492 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:12.538494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:12.538882 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:12.538883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:12.538945 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:12.539387 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:12.539392 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:12.541168 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
W20260809 23:15:12.541474 18339 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:12.536431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:48459 (local address 127.2.76.254:40469)
0809 23:15:12.536472 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:12.536474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:12.536595 (+ 121us) server_negotiation.cc:407] Connection header received
0809 23:15:12.536699 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:12.536700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:12.536715 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:12.536764 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:12.537191 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:12.537799 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:12.538569 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:12.538791 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:12.538834 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:12.541407 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:48459: BlockingRecv error: recv got EOF from 127.2.76.195:48459 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 09 23:15:13 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:13 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:13.324196 18345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:13.319715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:45119)
0809 23:15:13.319862 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:13.319873 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:13.319925 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:13.320080 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:13.320081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:13.320320 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:13.320368 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:13.320370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:13.320960 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:13.320960 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:13.321453 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:13.321455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:13.321785 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:13.321786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:13.321835 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:13.322285 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:13.322290 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:13.324087 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":54,"threads_started":1}
W20260809 23:15:13.324312 18346 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:13.319862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:45119 (local address 127.2.76.254:40469)
0809 23:15:13.319965 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:13.319967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:13.319974 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:15:13.320000 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:13.320001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:13.320009 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:13.320061 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:13.320434 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:13.320896 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:13.321537 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:13.321728 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:13.321777 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:13.324218 (+ 2441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:45119: BlockingRecv error: recv got EOF from 127.2.76.193:45119 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":55,"threads_started":1}
Aug 09 23:15:13 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:13 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:13.434769 18347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:13.430453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:36381)
0809 23:15:13.430562 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:13.430572 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:13.430643 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:13.430776 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:13.430776 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:13.430921 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:13.430964 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:13.430966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:13.431608 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:13.431608 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:13.432072 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:13.432073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:13.432499 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:13.432501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:13.432546 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:13.432964 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:13.432969 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:13.434716 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":59,"threads_started":1}
W20260809 23:15:13.434914 18346 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:13.430497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:36381 (local address 127.2.76.254:40469)
0809 23:15:13.430538 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:13.430539 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:13.430618 (+ 79us) server_negotiation.cc:407] Connection header received
0809 23:15:13.430705 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:13.430706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:13.430718 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:13.430768 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:13.431021 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:13.431549 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:13.432240 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:13.432418 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:13.432484 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:13.434798 (+ 2314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:36381: BlockingRecv error: recv got EOF from 127.2.76.194:36381 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 23:15:13 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:13 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:13.545876 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:13.541630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:51553)
0809 23:15:13.541791 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:13.541799 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:13.541878 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:13.542014 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:13.542015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:13.542188 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:13.542233 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:13.542234 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:13.542826 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:13.542828 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:13.543293 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:13.543294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:13.543598 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:13.543599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:13.543655 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:13.544083 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:13.544118 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:13.545824 (+ 1706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":57,"threads_started":1}
W20260809 23:15:13.545986 18346 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:13.541785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:51553 (local address 127.2.76.254:40469)
0809 23:15:13.541860 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:13.541863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:13.541871 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:15:13.541933 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:13.541934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:13.541946 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:13.542014 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:13.542299 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:13.542739 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:13.543367 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:13.543535 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:13.543598 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:13.545916 (+ 2318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:51553: BlockingRecv error: recv got EOF from 127.2.76.195:51553 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 23:15:14 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:14 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:14.328577 18352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:14.324581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:40179)
0809 23:15:14.324691 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:14.324703 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:14.324767 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:14.324881 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:14.324881 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:14.325032 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:14.325081 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:14.325082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:14.325624 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:14.325625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:14.326062 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:14.326063 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:14.326333 (+ 270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:14.326334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:14.326366 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:14.326772 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:14.326776 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:14.328466 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":53,"threads_started":1}
W20260809 23:15:14.328635 18353 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:14.324655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:40179 (local address 127.2.76.254:40469)
0809 23:15:14.324773 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:14.324774 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:14.324784 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:15:14.324807 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:14.324807 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:14.324816 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:14.324864 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:14.325136 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:14.325569 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:14.326120 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:14.326282 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:14.326323 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:14.328564 (+ 2241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:40179: BlockingRecv error: recv got EOF from 127.2.76.193:40179 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
Aug 09 23:15:14 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:14 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:14.439900 18354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:14.435256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:40261)
0809 23:15:14.435465 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:14.435493 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:14.435628 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:14.435857 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:14.435858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:14.436156 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:14.436226 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:14.436228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:14.436923 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:14.436924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:14.437376 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:14.437377 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:14.437682 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:14.437682 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:14.437717 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:14.438145 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:14.438149 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:14.439845 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":78,"threads_started":1}
W20260809 23:15:14.439996 18353 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:14.435438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:40261 (local address 127.2.76.254:40469)
0809 23:15:14.435543 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:14.435545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:14.435593 (+ 48us) server_negotiation.cc:407] Connection header received
0809 23:15:14.435755 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:14.435756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:14.435770 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:14.435835 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:14.436338 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:14.436846 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:14.437436 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:14.437602 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:14.437674 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:14.439917 (+ 2243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:40261: BlockingRecv error: recv got EOF from 127.2.76.194:40261 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 23:15:14 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:14 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:14.550453 18355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:14.546276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:57769)
0809 23:15:14.546422 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:14.546431 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:14.546499 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:14.546644 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:14.546645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:14.546804 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:14.546849 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:14.546851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:14.547453 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:14.547455 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:14.547924 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:14.547925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:14.548328 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:14.548329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:14.548393 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:14.548793 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:14.548798 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:14.550403 (+ 1605us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
W20260809 23:15:14.550552 18353 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:14.546346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:57769 (local address 127.2.76.254:40469)
0809 23:15:14.546402 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:14.546404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:14.546482 (+ 78us) server_negotiation.cc:407] Connection header received
0809 23:15:14.546563 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:14.546564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:14.546576 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:14.546638 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:14.546915 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:14.547378 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:14.548023 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:14.548249 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:14.548325 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:14.550485 (+ 2160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:57769: BlockingRecv error: recv got EOF from 127.2.76.195:57769 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 09 23:15:15 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:15 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:15.333180 18356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:15.328957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:41961)
0809 23:15:15.329078 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:15.329087 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:15.329128 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:15.329267 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:15.329268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:15.329422 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:15.329469 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:15.329470 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:15.330036 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:15.330037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:15.330515 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:15.330517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:15.330791 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:15.330792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:15.330829 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:15.331423 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:15.331429 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:15.333102 (+ 1673us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
W20260809 23:15:15.333257 18357 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:15.329006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:41961 (local address 127.2.76.254:40469)
0809 23:15:15.329117 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:15.329118 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:15.329125 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:15:15.329181 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:15.329181 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:15.329191 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:15.329254 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:15.329533 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:15.329984 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:15.330576 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:15.330738 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:15.330781 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:15.333194 (+ 2413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:41961: BlockingRecv error: recv got EOF from 127.2.76.193:41961 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":63,"threads_started":1}
Aug 09 23:15:15 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:15 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:15.444231 18358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:15.440303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:34023)
0809 23:15:15.440413 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:15.440423 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:15.440479 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:15.440585 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:15.440586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:15.440726 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:15.440768 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:15.440769 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:15.441292 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:15.441293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:15.441793 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:15.441795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:15.442070 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:15.442071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:15.442116 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:15.442501 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:15.442505 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:15.444148 (+ 1643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":66,"threads_started":1}
W20260809 23:15:15.444264 18357 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:15.440350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:34023 (local address 127.2.76.254:40469)
0809 23:15:15.440399 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:15.440405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:15.440475 (+ 70us) server_negotiation.cc:407] Connection header received
0809 23:15:15.440519 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:15.440520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:15.440529 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:15.440572 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:15.440816 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:15.441232 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:15.441849 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:15.442010 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:15.442056 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:15.444209 (+ 2153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:34023: BlockingRecv error: recv got EOF from 127.2.76.194:34023 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 23:15:15 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:15 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:15.554908 18359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:15.551033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:53665)
0809 23:15:15.551179 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:15.551187 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:15.551236 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:15.551335 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:15.551336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:15.551476 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:15.551529 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:15.551531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:15.552049 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:15.552050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:15.552517 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:15.552519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:15.552806 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:15.552807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:15.552879 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:15.553343 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:15.553347 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:15.554848 (+ 1501us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
W20260809 23:15:15.555011 18357 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:15.551131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:53665 (local address 127.2.76.254:40469)
0809 23:15:15.551181 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:15.551182 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:15.551227 (+ 45us) server_negotiation.cc:407] Connection header received
0809 23:15:15.551275 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:15.551275 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:15.551284 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:15.551326 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:15.551581 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:15.551994 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:15.552578 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:15.552731 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:15.552779 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:15.554930 (+ 2151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:53665: BlockingRecv error: recv got EOF from 127.2.76.195:53665 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 23:15:16 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:16 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:16.344686 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:16.333696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:59257)
0809 23:15:16.333819 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:16.333829 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:16.333877 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:16.334005 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:16.334017 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:16.334182 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:16.334241 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:16.334242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:16.334836 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:16.334836 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:16.335642 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:16.335646 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:16.335982 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:16.335983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:16.336064 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:16.342918 (+ 6854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:16.342922 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:16.344616 (+ 1694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
W20260809 23:15:16.344805 18363 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:16.333736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:59257 (local address 127.2.76.254:40469)
0809 23:15:16.333856 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:16.333857 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:16.333866 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:15:16.333931 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:16.333932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:16.333942 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:16.334000 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:16.334294 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:16.334772 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:16.335728 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:16.335918 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:16.335961 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:16.344707 (+ 8746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:59257: BlockingRecv error: recv got EOF from 127.2.76.193:59257 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":65,"threads_started":1}
Aug 09 23:15:16 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:16 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:16.448736 18364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:16.444630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:47935)
0809 23:15:16.444737 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:16.444747 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:16.444798 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:16.444908 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:16.444909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:16.445062 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:16.445105 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:16.445106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:16.445636 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:16.445637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:16.446091 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:16.446092 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:16.446402 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:16.446403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:16.446447 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:16.446827 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:16.446832 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:16.448634 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":53,"threads_started":1}
W20260809 23:15:16.448791 18363 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:16.444686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:47935 (local address 127.2.76.254:40469)
0809 23:15:16.444750 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:16.444752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:16.444797 (+ 45us) server_negotiation.cc:407] Connection header received
0809 23:15:16.444846 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:16.444846 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:16.444856 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:16.444890 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:16.445156 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:16.445582 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:16.446149 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:16.446303 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:16.446345 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:16.448717 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:47935: BlockingRecv error: recv got EOF from 127.2.76.194:47935 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 23:15:16 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:16 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:16.559473 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:16.555524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:38667)
0809 23:15:16.555653 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:16.555664 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:16.555725 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:16.555874 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:16.555875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:16.556079 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:16.556138 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:16.556140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:16.556671 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:16.556671 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:16.557161 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:16.557162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:16.557423 (+ 261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:16.557424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:16.557461 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:16.557823 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:16.557827 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:16.559407 (+ 1580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
W20260809 23:15:16.559533 18363 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:16.555549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:38667 (local address 127.2.76.254:40469)
0809 23:15:16.555605 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:16.555607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:16.555722 (+ 115us) server_negotiation.cc:407] Connection header received
0809 23:15:16.555774 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:16.555775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:16.555790 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:16.555861 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:16.556203 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:16.556617 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:16.557218 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:16.557368 (+ 150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:16.557411 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:16.559488 (+ 2077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:38667: BlockingRecv error: recv got EOF from 127.2.76.195:38667 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 23:15:17 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:17 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:17.349504 18366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:17.345115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:45757)
0809 23:15:17.345279 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:17.345290 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:17.345351 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:17.345481 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:17.345482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:17.345658 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:17.345706 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:17.345707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:17.346376 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:17.346377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:17.346966 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:17.346968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:17.347271 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:17.347272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:17.347309 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:17.347731 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:17.347735 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:17.349434 (+ 1699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
W20260809 23:15:17.349633 18367 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:17.345165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:45757 (local address 127.2.76.254:40469)
0809 23:15:17.345305 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:17.345307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:17.345335 (+ 28us) server_negotiation.cc:407] Connection header received
0809 23:15:17.345405 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:17.345406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:17.345417 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:17.345467 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:17.345763 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:17.346316 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:17.347031 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:17.347212 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:17.347262 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:17.349528 (+ 2266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:45757: BlockingRecv error: recv got EOF from 127.2.76.193:45757 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
Aug 09 23:15:17 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:17 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:17.453615 18369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:17.449479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:52447)
0809 23:15:17.449614 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:17.449624 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:17.449675 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:17.449789 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:17.449789 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:17.449944 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:17.449988 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:17.449990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:17.450530 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:17.450531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:17.451000 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:17.451001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:17.451285 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:17.451286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:17.451322 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:17.451716 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:17.451720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:17.453552 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
W20260809 23:15:17.453689 18367 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:17.449559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:52447 (local address 127.2.76.254:40469)
0809 23:15:17.449613 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:17.449615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:17.449673 (+ 58us) server_negotiation.cc:407] Connection header received
0809 23:15:17.449715 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:17.449716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:17.449726 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:17.449775 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:17.450039 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:17.450473 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:17.451065 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:17.451229 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:17.451273 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:17.453636 (+ 2363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:52447: BlockingRecv error: recv got EOF from 127.2.76.194:52447 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 23:15:17 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:17 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:17.564313 18370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:17.559902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:50509)
0809 23:15:17.560021 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:17.560030 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:17.560085 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:17.560243 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:17.560243 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:17.560418 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:17.560466 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:17.560467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:17.561141 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:17.561142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:17.561703 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:17.561705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:17.562042 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:17.562043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:17.562080 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:17.562475 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:17.562479 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:17.564253 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
W20260809 23:15:17.564369 18367 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:17.559951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:50509 (local address 127.2.76.254:40469)
0809 23:15:17.560011 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:17.560013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:17.560070 (+ 57us) server_negotiation.cc:407] Connection header received
0809 23:15:17.560177 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:17.560178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:17.560195 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:17.560233 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:17.560521 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:17.561078 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:17.561760 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:17.561977 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:17.562044 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:17.564326 (+ 2282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:50509: BlockingRecv error: recv got EOF from 127.2.76.195:50509 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 09 23:15:18 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:18 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:18.354053 18371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:18.349895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:58819)
0809 23:15:18.350048 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:18.350057 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:18.350106 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:18.350214 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:18.350215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:18.350364 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:18.350408 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:18.350409 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:18.350938 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:18.350938 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:18.351421 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:18.351422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:18.351821 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:18.351822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:18.351884 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:18.352366 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:18.352371 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:18.353963 (+ 1592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
W20260809 23:15:18.354108 18372 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:18.349940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:58819 (local address 127.2.76.254:40469)
0809 23:15:18.350051 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:18.350053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:18.350103 (+ 50us) server_negotiation.cc:407] Connection header received
0809 23:15:18.350144 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:18.350144 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:18.350154 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:18.350204 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:18.350451 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:18.350880 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:18.351478 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:18.351715 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:18.351804 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:18.354042 (+ 2238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:58819: BlockingRecv error: recv got EOF from 127.2.76.193:58819 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":63,"threads_started":1}
Aug 09 23:15:18 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:18 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:18.458067 18373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:18.454156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:49985)
0809 23:15:18.454278 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:18.454291 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:18.454338 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:18.454447 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:18.454448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:18.454589 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:18.454631 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:18.454632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:18.455155 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:18.455155 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:18.455611 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:18.455612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:18.455913 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:18.455914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:18.455949 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:18.456392 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:18.456396 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:18.458000 (+ 1604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
W20260809 23:15:18.458148 18372 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:18.454240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:49985 (local address 127.2.76.254:40469)
0809 23:15:18.454295 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:18.454296 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:18.454339 (+ 43us) server_negotiation.cc:407] Connection header received
0809 23:15:18.454378 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:18.454378 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:18.454387 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:18.454431 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:18.454682 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:18.455099 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:18.455670 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:18.455856 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:18.455897 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:18.458073 (+ 2176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:49985: BlockingRecv error: recv got EOF from 127.2.76.194:49985 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 09 23:15:18 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:18 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:18.569222 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:18.564728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:50059)
0809 23:15:18.564864 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:18.564875 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:18.564951 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:18.565112 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:18.565113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:18.565327 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:18.565399 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:18.565401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:18.566068 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:18.566069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:18.566612 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:18.566613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:18.566948 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:18.566948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:18.567009 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:18.567457 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:18.567463 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:18.569158 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
W20260809 23:15:18.569281 18372 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:18.564787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:50059 (local address 127.2.76.254:40469)
0809 23:15:18.564859 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:18.564861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:18.564934 (+ 73us) server_negotiation.cc:407] Connection header received
0809 23:15:18.565021 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:18.565022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:18.565034 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:18.565105 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:18.565473 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:18.565981 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:18.566703 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:18.566882 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:18.566950 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:18.569237 (+ 2287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:50059: BlockingRecv error: recv got EOF from 127.2.76.195:50059 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 23:15:19 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:19 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:19.359019 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:19.354585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:58315)
0809 23:15:19.354716 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:19.354726 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:19.354777 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:19.354878 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:19.354879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:19.355041 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:19.355087 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:19.355088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:19.355632 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:19.355632 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:19.356167 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:19.356170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:19.356443 (+ 273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:19.356444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:19.356481 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:19.356962 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:19.356967 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:19.358919 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
W20260809 23:15:19.359081 18381 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:19.354655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:58315 (local address 127.2.76.254:40469)
0809 23:15:19.354772 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:19.354774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:19.354783 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:15:19.354814 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:19.354815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:19.354823 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:19.354868 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:19.355139 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:19.355575 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:19.356228 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:19.356390 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:19.356433 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:19.359007 (+ 2574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:58315: BlockingRecv error: recv got EOF from 127.2.76.193:58315 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
Aug 09 23:15:19 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:19 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:19.462824 18382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:19.458490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:52885)
0809 23:15:19.458681 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:19.458690 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:19.458743 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:19.458857 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:19.458858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:19.459024 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:19.459071 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:19.459072 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:19.459646 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:19.459647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:19.460155 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:19.460157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:19.460460 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:19.460461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:19.460522 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:19.461004 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:19.461008 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:19.462745 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":99,"threads_started":1}
W20260809 23:15:19.462852 18381 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:19.458579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:52885 (local address 127.2.76.254:40469)
0809 23:15:19.458649 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:19.458651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:19.458741 (+ 90us) server_negotiation.cc:407] Connection header received
0809 23:15:19.458797 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:19.458798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:19.458808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:19.458846 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:19.459123 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:19.459583 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:19.460219 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:19.460391 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:19.460433 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:19.462809 (+ 2376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:52885: BlockingRecv error: recv got EOF from 127.2.76.194:52885 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 23:15:19 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:19 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.195@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:19.573613 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:19.569725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.195:32815)
0809 23:15:19.569854 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:19.569864 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:19.569923 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:19.570032 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:19.570032 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:19.570183 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:19.570231 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:19.570232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:19.570763 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:19.570763 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:19.571202 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:19.571203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:19.571463 (+ 260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:19.571464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:19.571505 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:19.571932 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:19.571936 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:19.573546 (+ 1610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
W20260809 23:15:19.573709 18381 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:19.569788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:32815 (local address 127.2.76.254:40469)
0809 23:15:19.569837 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:19.569839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:19.569916 (+ 77us) server_negotiation.cc:407] Connection header received
0809 23:15:19.569961 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:19.569961 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:19.569970 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:19.570019 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:19.570281 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:19.570706 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:19.571257 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:19.571409 (+ 152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:19.571450 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:19.573642 (+ 2192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.195:32815: BlockingRecv error: recv got EOF from 127.2.76.195:32815 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 23:15:20 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:20 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.193@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:20.364274 18385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:20.359658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.193:45397)
0809 23:15:20.359797 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:20.359806 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:20.359852 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:20.359979 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:20.359980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:20.360183 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:20.360249 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:20.360250 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.360887 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.360888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:20.361411 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:20.361430 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.361894 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.361895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:20.361942 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:20.362392 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:20.362397 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:20.364219 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
W20260809 23:15:20.364373 18386 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:20.359733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:45397 (local address 127.2.76.254:40469)
0809 23:15:20.359827 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:20.359829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:20.359838 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:15:20.359906 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:20.359906 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:20.359916 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:20.359968 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:20.360304 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.360818 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.361515 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.361828 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.361883 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:20.364302 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.193:45397: BlockingRecv error: recv got EOF from 127.2.76.193:45397 (error 108)
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":46,"threads_started":1}
Aug 09 23:15:20 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:15:20 dist-test-slave-7vjv krb5kdc[16556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.76.194@KRBTEST.COM for oryx/127.2.76.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:15:20.467291 18387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:20.463294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40469 (local address 127.2.76.194:51959)
0809 23:15:20.463403 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:20.463412 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:20.463463 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:20.463571 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:20.463571 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:20.463712 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:20.463783 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:20.463785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.464398 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.464399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:20.464848 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:20.464850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.465119 (+ 269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.465120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:20.465155 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:20.465509 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:20.465513 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:20.467210 (+ 1697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.76.254:40469: Server oryx/127.2.76.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
W20260809 23:15:20.467401 18386 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:20.463343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:51959 (local address 127.2.76.254:40469)
0809 23:15:20.463419 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:20.463421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:20.463450 (+ 29us) server_negotiation.cc:407] Connection header received
0809 23:15:20.463513 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:20.463514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:20.463522 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:20.463558 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:20.463849 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.464337 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.464906 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.465068 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.465111 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:20.467346 (+ 2235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.76.194:51959: BlockingRecv error: recv got EOF from 127.2.76.194:51959 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 23:15:20.509909 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 17583
I20260809 23:15:20.523878 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 17844
I20260809 23:15:20.533527 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 18071
I20260809 23:15:20.543174 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 17466
2026-08-09T23:15:20Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17423 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T23:15:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:15:20Z Disabled control of system clock
I20260809 23:15:20.574432 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:36601
--webserver_interface=127.2.76.254
--webserver_port=0
--builtin_ntp_servers=127.2.76.212:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:36601
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 23:15:20.645942 18396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:20.646099 18396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:20.646117 18396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:20.647121 18396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:15:20.647156 18396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:20.647168 18396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:20.647181 18396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:15:20.647193 18396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:15:20.648425 18396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:36601
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:36601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:20.648589 18396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:20.648782 18396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:20.652904 18410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:20.653007 18396 server_base.cc:1034] running on GCE node
W20260809 23:15:20.652949 18402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:20.652904 18405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:20.653380 18396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:20.653664 18396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:20.654821 18396 hybrid_clock.cc:648] HybridClock initialized: now 1786317320654813 us; error 34 us; skew 500 ppm
I20260809 23:15:20.655903 18396 webserver.cc:460] Webserver started at http://127.2.76.254:36573/ using document root <none> and password file <none>
I20260809 23:15:20.656183 18396 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:20.656258 18396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:20.656381 18396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:20.657315 18396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "fc6c1fb3a3e8412bb3716761f595b32d"
format_stamp: "Formatted at 2026-08-09 23:15:20 on dist-test-slave-7vjv"
server_key: "1b9ad2f8de79db809aeb32ea54148fad"
server_key_iv: "624b34ce0423ddbafb5f581b3101ff11"
server_key_version: "encryptionkey@0"
I20260809 23:15:20.657670 18396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "fc6c1fb3a3e8412bb3716761f595b32d"
format_stamp: "Formatted at 2026-08-09 23:15:20 on dist-test-slave-7vjv"
server_key: "1b9ad2f8de79db809aeb32ea54148fad"
server_key_iv: "624b34ce0423ddbafb5f581b3101ff11"
server_key_version: "encryptionkey@0"
I20260809 23:15:20.659134 18396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:15:20.659772 18420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:20.659922 18396 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:15:20.660008 18396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "fc6c1fb3a3e8412bb3716761f595b32d"
format_stamp: "Formatted at 2026-08-09 23:15:20 on dist-test-slave-7vjv"
server_key: "1b9ad2f8de79db809aeb32ea54148fad"
server_key_iv: "624b34ce0423ddbafb5f581b3101ff11"
server_key_version: "encryptionkey@0"
I20260809 23:15:20.660089 18396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:20.679875 18396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:20.680184 18396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:20.680313 18396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:20.683686 18396 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:36601
I20260809 23:15:20.683733 18509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:36601 every 8 connection(s)
I20260809 23:15:20.684010 18396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:15:20.684585 18511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:20.686664 18511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d: Bootstrap starting.
I20260809 23:15:20.687232 18511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:20.687496 18511 log.cc:824] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:15:20.688068 18511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d: No bootstrap required, opened a new log
I20260809 23:15:20.688496 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 18396
I20260809 23:15:20.688576 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:15:20.688766 2355 external_mini_cluster.cc:1428] Setting key 31b0f8d2f453f1aab0c118c07e3ea587
I20260809 23:15:20.690003 18511 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6c1fb3a3e8412bb3716761f595b32d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36601 } }
I20260809 23:15:20.690150 18511 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:20.690181 18511 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6c1fb3a3e8412bb3716761f595b32d, State: Initialized, Role: FOLLOWER
I20260809 23:15:20.690323 18511 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6c1fb3a3e8412bb3716761f595b32d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36601 } }
I20260809 23:15:20.690414 18511 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:15:20.690454 18511 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:15:20.690496 18511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:20.690937 18511 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6c1fb3a3e8412bb3716761f595b32d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36601 } }
I20260809 23:15:20.691013 18511 leader_election.cc:302] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc6c1fb3a3e8412bb3716761f595b32d; no voters:
I20260809 23:15:20.691185 18511 leader_election.cc:288] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:15:20.691213 18518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:15:20.691316 18518 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [term 1 LEADER]: Becoming Leader. State: Replica: fc6c1fb3a3e8412bb3716761f595b32d, State: Running, Role: LEADER
I20260809 23:15:20.691412 18511 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:15:20.691442 18518 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6c1fb3a3e8412bb3716761f595b32d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36601 } }
I20260809 23:15:20.691819 18518 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc6c1fb3a3e8412bb3716761f595b32d. Latest consensus state: current_term: 1 leader_uuid: "fc6c1fb3a3e8412bb3716761f595b32d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6c1fb3a3e8412bb3716761f595b32d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36601 } } }
I20260809 23:15:20.691828 18525 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc6c1fb3a3e8412bb3716761f595b32d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6c1fb3a3e8412bb3716761f595b32d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36601 } } }
I20260809 23:15:20.691982 18518 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [sys.catalog]: This master's current role is: LEADER
I20260809 23:15:20.692209 18525 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d [sys.catalog]: This master's current role is: LEADER
W20260809 23:15:20.693168 18551 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:15:20.693260 18551 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:15:20.693344 18541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:15:20.693475 18541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:15:20.694386 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:20.689210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.2.76.254:36601)
0809 23:15:20.689402 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:20.689408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:20.689429 (+ 21us) server_negotiation.cc:407] Connection header received
0809 23:15:20.689986 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:20.689991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:20.690059 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:20.690171 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:20.690385 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.691678 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.692632 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.693103 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.693222 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:20.693793 (+ 571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:20.693796 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:20.693797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:15:20.693945 (+ 148us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:20.693975 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:15:20.693984 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:20.694057 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:20.694121 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 23:15:20.694188 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
I20260809 23:15:20.694712 18541 catalog_manager.cc:1347] Generated new cluster ID: 2d5063e95c26431bb9cbe68259429500
I20260809 23:15:20.694759 18541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:15:20.699049 18541 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:15:20.699576 18541 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:15:20.703660 18541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc6c1fb3a3e8412bb3716761f595b32d: Generated new TSK 0
I20260809 23:15:20.703799 18541 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:15:20.706811 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36601
--builtin_ntp_servers=127.2.76.212:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 23:15:20.780931 18557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:20.781139 18557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:20.781170 18557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:20.782217 18557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:20.782275 18557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:20.782330 18557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:15:20.783489 18557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:20.783720 18557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:20.783969 18557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:20.788146 18571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:20.788157 18557 server_base.cc:1034] running on GCE node
W20260809 23:15:20.788120 18574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:20.788149 18569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:20.788450 18557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:20.788679 18557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:20.789819 18557 hybrid_clock.cc:648] HybridClock initialized: now 1786317320789805 us; error 29 us; skew 500 ppm
I20260809 23:15:20.790952 18557 webserver.cc:460] Webserver started at http://127.2.76.193:37265/ using document root <none> and password file <none>
I20260809 23:15:20.791193 18557 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:20.791251 18557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:20.791379 18557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:20.792460 18557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "a082e16d12aa4c9e940391ff6b629272"
format_stamp: "Formatted at 2026-08-09 23:15:20 on dist-test-slave-7vjv"
server_key: "3c2caa923ae497c463028c384200b91c"
server_key_iv: "71cfa9518b232b8189b13e8cc28bbfc6"
server_key_version: "encryptionkey@0"
I20260809 23:15:20.792829 18557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "a082e16d12aa4c9e940391ff6b629272"
format_stamp: "Formatted at 2026-08-09 23:15:20 on dist-test-slave-7vjv"
server_key: "3c2caa923ae497c463028c384200b91c"
server_key_iv: "71cfa9518b232b8189b13e8cc28bbfc6"
server_key_version: "encryptionkey@0"
I20260809 23:15:20.794126 18557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:15:20.794801 18588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:20.794931 18557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:15:20.795011 18557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "a082e16d12aa4c9e940391ff6b629272"
format_stamp: "Formatted at 2026-08-09 23:15:20 on dist-test-slave-7vjv"
server_key: "3c2caa923ae497c463028c384200b91c"
server_key_iv: "71cfa9518b232b8189b13e8cc28bbfc6"
server_key_version: "encryptionkey@0"
I20260809 23:15:20.795071 18557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:20.830663 18557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:20.830955 18557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:20.831089 18557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:20.831278 18557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:15:20.831559 18557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:20.831590 18557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:20.831632 18557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:20.831653 18557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:20.836906 18557 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:33493
I20260809 23:15:20.836952 18788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:33493 every 8 connection(s)
I20260809 23:15:20.837226 18557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:15:20.841432 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:20.837795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:41225 (local address 127.2.76.254:36601)
0809 23:15:20.837848 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:20.837850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:20.838024 (+ 174us) server_negotiation.cc:407] Connection header received
0809 23:15:20.838465 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:20.838467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:20.838480 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:20.838545 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:20.839138 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.839550 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.840352 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.840515 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.840558 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:20.841237 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:20.841238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:20.841238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:15:20.841244 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:20.841258 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:15:20.841261 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:20.841300 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:20.841362 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 23:15:20.841382 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 23:15:20.841596 18794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:20.837791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:36601 (local address 127.2.76.193:41225)
0809 23:15:20.837937 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:20.837963 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:20.838364 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:20.838645 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:20.838650 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:20.838660 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:20.839072 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:20.839077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.839625 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.839626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:20.840285 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:20.840287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.840574 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.840575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:20.840682 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:20.841130 (+ 448us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:15:20.841131 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:20.841154 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:15:20.841172 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:15:20.841307 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:15:20.841310 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:15:20.841312 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:15:20.841354 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 23:15:20.841421 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":71,"threads_started":1}
I20260809 23:15:20.842053 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 18557
I20260809 23:15:20.842063 18792 heartbeater.cc:347] Connected to a master server at 127.2.76.254:36601
I20260809 23:15:20.842142 18792 heartbeater.cc:464] Registering TS with master...
I20260809 23:15:20.842144 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:15:20.842307 18792 heartbeater.cc:511] Master 127.2.76.254:36601 requested a full tablet report, sending...
I20260809 23:15:20.842324 2355 external_mini_cluster.cc:1428] Setting key 160680b810cebdee4928a612682a9336
I20260809 23:15:20.842885 18456 ts_manager.cc:194] Registered new tserver with Master: a082e16d12aa4c9e940391ff6b629272 (127.2.76.193:33493)
I20260809 23:15:20.843240 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36601
--builtin_ntp_servers=127.2.76.212:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 23:15:20.843497 18456 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.76.193:41225
W20260809 23:15:20.918195 18795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:20.918396 18795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:20.918416 18795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:20.919545 18795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:20.919591 18795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:20.919634 18795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:15:20.920857 18795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:20.921075 18795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:20.921337 18795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:20.925689 18802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:20.925719 18805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:20.925815 18795 server_base.cc:1034] running on GCE node
W20260809 23:15:20.925715 18801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:20.926056 18795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:20.926265 18795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:20.927413 18795 hybrid_clock.cc:648] HybridClock initialized: now 1786317320927403 us; error 24 us; skew 500 ppm
I20260809 23:15:20.928551 18795 webserver.cc:460] Webserver started at http://127.2.76.194:35597/ using document root <none> and password file <none>
I20260809 23:15:20.928768 18795 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:20.928828 18795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:20.928948 18795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:20.929930 18795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "223c0390b8b349149b6e1bb32b29c930"
format_stamp: "Formatted at 2026-08-09 23:15:20 on dist-test-slave-7vjv"
server_key: "b9c65136a2103680bc0070f31ebc274b"
server_key_iv: "237c2679bde7839ef036fb3f68012038"
server_key_version: "encryptionkey@0"
I20260809 23:15:20.930311 18795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "223c0390b8b349149b6e1bb32b29c930"
format_stamp: "Formatted at 2026-08-09 23:15:20 on dist-test-slave-7vjv"
server_key: "b9c65136a2103680bc0070f31ebc274b"
server_key_iv: "237c2679bde7839ef036fb3f68012038"
server_key_version: "encryptionkey@0"
I20260809 23:15:20.931541 18795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:15:20.932181 18816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:20.932363 18795 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:15:20.932461 18795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "223c0390b8b349149b6e1bb32b29c930"
format_stamp: "Formatted at 2026-08-09 23:15:20 on dist-test-slave-7vjv"
server_key: "b9c65136a2103680bc0070f31ebc274b"
server_key_iv: "237c2679bde7839ef036fb3f68012038"
server_key_version: "encryptionkey@0"
I20260809 23:15:20.932540 18795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:20.958616 18795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:20.958839 18795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:20.958957 18795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:20.959138 18795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:15:20.959400 18795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:20.959444 18795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:20.959474 18795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:20.959496 18795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:20.965040 18795 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:44571
I20260809 23:15:20.965075 19004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:44571 every 8 connection(s)
I20260809 23:15:20.965354 18795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:15:20.968230 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 18795
I20260809 23:15:20.968333 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:15:20.968482 2355 external_mini_cluster.cc:1428] Setting key 93ec7b1c883a1caa962a5ad934960d61
I20260809 23:15:20.969602 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36601
--builtin_ntp_servers=127.2.76.212:45211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 23:15:20.970242 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:20.965955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:46351 (local address 127.2.76.254:36601)
0809 23:15:20.966017 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:20.966019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:20.966221 (+ 202us) server_negotiation.cc:407] Connection header received
0809 23:15:20.966638 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:20.966639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:20.966647 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:20.966684 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:20.967228 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.967676 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.968876 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.969084 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.969155 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:20.970008 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:20.970010 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:20.970010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:15:20.970014 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:20.970027 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:15:20.970031 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:20.970084 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:20.970171 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 23:15:20.970192 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 23:15:20.970496 19010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:20.965951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:36601 (local address 127.2.76.194:46351)
0809 23:15:20.966117 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:20.966149 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:20.966539 (+ 390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:20.966725 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:20.966728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:20.966751 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:20.967167 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:20.967170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.967741 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.967742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:20.968802 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:20.968805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:20.969163 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:20.969165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:20.969326 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:20.969847 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:15:20.969847 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:20.969880 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:15:20.969907 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:15:20.970099 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:15:20.970105 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:15:20.970107 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:15:20.970167 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 23:15:20.970262 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
I20260809 23:15:20.971643 19007 heartbeater.cc:347] Connected to a master server at 127.2.76.254:36601
I20260809 23:15:20.971740 19007 heartbeater.cc:464] Registering TS with master...
I20260809 23:15:20.971920 19007 heartbeater.cc:511] Master 127.2.76.254:36601 requested a full tablet report, sending...
I20260809 23:15:20.972301 18456 ts_manager.cc:194] Registered new tserver with Master: 223c0390b8b349149b6e1bb32b29c930 (127.2.76.194:44571)
I20260809 23:15:20.972645 18456 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.76.194:46351
W20260809 23:15:21.042413 19012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:21.042605 19012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:21.042637 19012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:21.043588 19012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:21.043648 19012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:21.043680 19012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:15:21.044821 19012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:21.045068 19012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:21.045320 19012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:21.049872 19021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:21.049983 19025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:21.049878 19020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:21.050163 19012 server_base.cc:1034] running on GCE node
I20260809 23:15:21.050379 19012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:21.050590 19012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:21.051734 19012 hybrid_clock.cc:648] HybridClock initialized: now 1786317321051712 us; error 32 us; skew 500 ppm
I20260809 23:15:21.052886 19012 webserver.cc:460] Webserver started at http://127.2.76.195:38581/ using document root <none> and password file <none>
I20260809 23:15:21.053110 19012 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:21.053177 19012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:21.053300 19012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:21.054304 19012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "bc3c8e4303c94f579e06dc4367c9f9a5"
format_stamp: "Formatted at 2026-08-09 23:15:21 on dist-test-slave-7vjv"
server_key: "169ce3bcc4c774e89a556238abbf310e"
server_key_iv: "51c1e2e760fa36840fdf73951a5ecdff"
server_key_version: "encryptionkey@0"
I20260809 23:15:21.054627 19012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "bc3c8e4303c94f579e06dc4367c9f9a5"
format_stamp: "Formatted at 2026-08-09 23:15:21 on dist-test-slave-7vjv"
server_key: "169ce3bcc4c774e89a556238abbf310e"
server_key_iv: "51c1e2e760fa36840fdf73951a5ecdff"
server_key_version: "encryptionkey@0"
I20260809 23:15:21.055775 19012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:15:21.056484 19040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.056634 19012 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.056718 19012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "bc3c8e4303c94f579e06dc4367c9f9a5"
format_stamp: "Formatted at 2026-08-09 23:15:21 on dist-test-slave-7vjv"
server_key: "169ce3bcc4c774e89a556238abbf310e"
server_key_iv: "51c1e2e760fa36840fdf73951a5ecdff"
server_key_version: "encryptionkey@0"
I20260809 23:15:21.056788 19012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:21.069111 19012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:21.069329 19012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:21.069447 19012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:21.069626 19012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:15:21.069898 19012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:21.069952 19012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.069981 19012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:21.070003 19012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.074924 19012 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:33873
I20260809 23:15:21.074973 19218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:33873 every 8 connection(s)
I20260809 23:15:21.075246 19012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:15:21.079834 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:21.075842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:57879 (local address 127.2.76.254:36601)
0809 23:15:21.075913 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:21.075915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:21.076145 (+ 230us) server_negotiation.cc:407] Connection header received
0809 23:15:21.076690 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:21.076691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:21.076704 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:21.076768 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:21.077399 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:21.077869 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:21.078723 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:21.078908 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:21.078973 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:21.079570 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:21.079571 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:21.079572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:15:21.079576 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:21.079588 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:15:21.079592 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:21.079635 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:21.079714 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 23:15:21.079742 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 23:15:21.080008 19223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:21.075784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:36601 (local address 127.2.76.195:57879)
0809 23:15:21.075993 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:21.076027 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:21.076583 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:21.076830 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:21.076834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:21.076843 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:21.077310 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:21.077313 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:21.077956 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:21.077958 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:21.078640 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:21.078642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:21.078981 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:21.078982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:21.079097 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:21.079443 (+ 346us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:15:21.079444 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:21.079465 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:15:21.079483 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:15:21.079640 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:15:21.079644 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:15:21.079647 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:15:21.079713 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 23:15:21.079796 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260809 23:15:21.080457 19219 heartbeater.cc:347] Connected to a master server at 127.2.76.254:36601
I20260809 23:15:21.080559 19219 heartbeater.cc:464] Registering TS with master...
I20260809 23:15:21.080675 19219 heartbeater.cc:511] Master 127.2.76.254:36601 requested a full tablet report, sending...
I20260809 23:15:21.080969 18456 ts_manager.cc:194] Registered new tserver with Master: bc3c8e4303c94f579e06dc4367c9f9a5 (127.2.76.195:33873)
I20260809 23:15:21.081278 18456 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.76.195:57879
I20260809 23:15:21.085383 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 19012
I20260809 23:15:21.085476 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:15:21.085630 2355 external_mini_cluster.cc:1428] Setting key 3cb6c996eeed5ec2b07f481281951b24
I20260809 23:15:21.086380 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 23:15:21.087906 18517 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:21.087512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40312 (local address 127.2.76.254:36601)
0809 23:15:21.087551 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:21.087552 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:21.087686 (+ 134us) server_negotiation.cc:407] Connection header received
0809 23:15:21.087729 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:21.087729 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:21.087738 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:21.087771 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:21.087869 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40312: BlockingRecv error: recv got EOF from 127.0.0.1:40312 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:15:21.087982 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 18557
I20260809 23:15:21.107770 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 18795
I20260809 23:15:21.117054 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 19012
I20260809 23:15:21.132885 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 18396
2026-08-09T23:15:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (586 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T23:15:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:15:21Z Disabled control of system clock
I20260809 23:15:21.161154 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:44937
--webserver_interface=127.2.76.254
--webserver_port=0
--builtin_ntp_servers=127.2.76.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:44937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 23:15:21.230397 19241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:21.230621 19241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:21.230660 19241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:21.231758 19241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:15:21.231822 19241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:21.231844 19241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:21.231866 19241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:15:21.231902 19241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:15:21.233057 19241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:44937
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:44937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:21.233330 19241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:21.233592 19241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:21.237814 19256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:21.237901 19250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:21.237907 19249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:21.238062 19241 server_base.cc:1034] running on GCE node
I20260809 23:15:21.238255 19241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:21.238446 19241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:21.239635 19241 hybrid_clock.cc:648] HybridClock initialized: now 1786317321239604 us; error 45 us; skew 500 ppm
I20260809 23:15:21.240782 19241 webserver.cc:460] Webserver started at http://127.2.76.254:38849/ using document root <none> and password file <none>
I20260809 23:15:21.240983 19241 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:21.241044 19241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:21.241173 19241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:21.242066 19241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "2170a1d8bfb74fcb89fffd9febed64d6"
format_stamp: "Formatted at 2026-08-09 23:15:21 on dist-test-slave-7vjv"
server_key: "64b474bcffd3e1186150840983b08aee"
server_key_iv: "e3110e123e5ab3625bb224b0a941d497"
server_key_version: "encryptionkey@0"
I20260809 23:15:21.242370 19241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "2170a1d8bfb74fcb89fffd9febed64d6"
format_stamp: "Formatted at 2026-08-09 23:15:21 on dist-test-slave-7vjv"
server_key: "64b474bcffd3e1186150840983b08aee"
server_key_iv: "e3110e123e5ab3625bb224b0a941d497"
server_key_version: "encryptionkey@0"
I20260809 23:15:21.243547 19241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:15:21.244153 19266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.244298 19241 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.244383 19241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "2170a1d8bfb74fcb89fffd9febed64d6"
format_stamp: "Formatted at 2026-08-09 23:15:21 on dist-test-slave-7vjv"
server_key: "64b474bcffd3e1186150840983b08aee"
server_key_iv: "e3110e123e5ab3625bb224b0a941d497"
server_key_version: "encryptionkey@0"
I20260809 23:15:21.244454 19241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:21.247267 19241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:21.247519 19241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:21.247659 19241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:21.251412 19241 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:44937
I20260809 23:15:21.251439 19341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:44937 every 8 connection(s)
I20260809 23:15:21.251734 19241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:15:21.252331 19344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:21.254990 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 19241
I20260809 23:15:21.255079 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:15:21.255106 19344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6: Bootstrap starting.
I20260809 23:15:21.255256 2355 external_mini_cluster.cc:1428] Setting key 4e9e5e96d5f9cb324b7aae23a99aa0c4
I20260809 23:15:21.255762 19344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:21.256085 19344 log.cc:824] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6: Log is configured to *not* fsync() on all Append() calls
I20260809 23:15:21.256798 19344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6: No bootstrap required, opened a new log
I20260809 23:15:21.257479 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:21.255706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40248 (local address 127.2.76.254:44937)
0809 23:15:21.255882 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:21.255889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:21.255915 (+ 26us) server_negotiation.cc:407] Connection header received
0809 23:15:21.256343 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:21.256349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:21.256390 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:21.256511 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:21.257108 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:21.257111 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:21.257111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:15:21.257115 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:21.257161 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:15:21.257170 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:21.257244 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:21.257300 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 23:15:21.257348 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260809 23:15:21.258420 19344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2170a1d8bfb74fcb89fffd9febed64d6" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44937 } }
I20260809 23:15:21.258553 19344 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:21.258595 19344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2170a1d8bfb74fcb89fffd9febed64d6, State: Initialized, Role: FOLLOWER
I20260809 23:15:21.258699 19344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2170a1d8bfb74fcb89fffd9febed64d6" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44937 } }
I20260809 23:15:21.258795 19344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:15:21.258842 19344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:15:21.258890 19344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:21.259661 19344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2170a1d8bfb74fcb89fffd9febed64d6" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44937 } }
I20260809 23:15:21.259771 19344 leader_election.cc:302] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2170a1d8bfb74fcb89fffd9febed64d6; no voters:
I20260809 23:15:21.259935 19344 leader_election.cc:288] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:15:21.259995 19354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:15:21.260166 19354 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [term 1 LEADER]: Becoming Leader. State: Replica: 2170a1d8bfb74fcb89fffd9febed64d6, State: Running, Role: LEADER
I20260809 23:15:21.260221 19344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:15:21.260440 19354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2170a1d8bfb74fcb89fffd9febed64d6" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44937 } }
I20260809 23:15:21.260788 19357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2170a1d8bfb74fcb89fffd9febed64d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2170a1d8bfb74fcb89fffd9febed64d6" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44937 } } }
I20260809 23:15:21.260777 19360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2170a1d8bfb74fcb89fffd9febed64d6. Latest consensus state: current_term: 1 leader_uuid: "2170a1d8bfb74fcb89fffd9febed64d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2170a1d8bfb74fcb89fffd9febed64d6" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 44937 } } }
I20260809 23:15:21.260883 19357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [sys.catalog]: This master's current role is: LEADER
I20260809 23:15:21.260888 19360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6 [sys.catalog]: This master's current role is: LEADER
I20260809 23:15:21.261129 19372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:15:21.261565 19372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:15:21.262813 19372 catalog_manager.cc:1347] Generated new cluster ID: b00df7e284234c91a2bd1c23acced90a
I20260809 23:15:21.262902 19372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:15:21.276557 19372 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:15:21.276701 19372 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:15:21.280920 19372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2170a1d8bfb74fcb89fffd9febed64d6: Generated new TSK 0
I20260809 23:15:21.281071 19372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:15:21.284324 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:44937
--builtin_ntp_servers=127.2.76.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 23:15:21.357914 19390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:21.358079 19390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:21.358098 19390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:21.359081 19390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:21.359115 19390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:21.359138 19390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:15:21.360545 19390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:44937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:21.360756 19390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:21.361043 19390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:21.365376 19399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:21.365463 19396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:21.365372 19397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:21.365466 19390 server_base.cc:1034] running on GCE node
I20260809 23:15:21.365805 19390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:21.366047 19390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:21.367188 19390 hybrid_clock.cc:648] HybridClock initialized: now 1786317321367181 us; error 25 us; skew 500 ppm
I20260809 23:15:21.373063 19390 webserver.cc:460] Webserver started at http://127.2.76.193:33181/ using document root <none> and password file <none>
I20260809 23:15:21.373272 19390 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:21.373311 19390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:21.373417 19390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:21.374363 19390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "2934b98199d14fce87fa5178d6dd542e"
format_stamp: "Formatted at 2026-08-09 23:15:21 on dist-test-slave-7vjv"
server_key: "6f93f02df97160c87f81e04a32403b69"
server_key_iv: "4366204f7aa6e053fd154265557dfe30"
server_key_version: "encryptionkey@0"
I20260809 23:15:21.374684 19390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "2934b98199d14fce87fa5178d6dd542e"
format_stamp: "Formatted at 2026-08-09 23:15:21 on dist-test-slave-7vjv"
server_key: "6f93f02df97160c87f81e04a32403b69"
server_key_iv: "4366204f7aa6e053fd154265557dfe30"
server_key_version: "encryptionkey@0"
I20260809 23:15:21.375941 19390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:15:21.376631 19406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.376784 19390 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.376878 19390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "2934b98199d14fce87fa5178d6dd542e"
format_stamp: "Formatted at 2026-08-09 23:15:21 on dist-test-slave-7vjv"
server_key: "6f93f02df97160c87f81e04a32403b69"
server_key_iv: "4366204f7aa6e053fd154265557dfe30"
server_key_version: "encryptionkey@0"
I20260809 23:15:21.376948 19390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:21.379746 19390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:21.379981 19390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:21.380136 19390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:21.380339 19390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:15:21.380620 19390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:21.380664 19390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.380697 19390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:21.380720 19390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.385932 19390 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:43595
I20260809 23:15:21.385977 19589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:43595 every 8 connection(s)
I20260809 23:15:21.386323 19390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:15:21.388674 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:21.386876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:54151 (local address 127.2.76.254:44937)
0809 23:15:21.386924 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:21.386926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:21.387141 (+ 215us) server_negotiation.cc:407] Connection header received
0809 23:15:21.387472 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:21.387473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:21.387478 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:21.387525 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:21.388488 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:21.388489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:21.388489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:15:21.388493 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:21.388513 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:15:21.388517 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:21.388560 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:21.388616 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 23:15:21.388627 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 23:15:21.388819 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:21.386884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:44937 (local address 127.2.76.193:54151)
0809 23:15:21.387037 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:21.387064 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:21.387403 (+ 339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:21.387572 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:21.387575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:21.387582 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:21.388364 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:15:21.388365 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:21.388389 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:15:21.388413 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:15:21.388560 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:15:21.388562 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:15:21.388564 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:15:21.388611 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 23:15:21.388650 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
I20260809 23:15:21.389277 19590 heartbeater.cc:347] Connected to a master server at 127.2.76.254:44937
I20260809 23:15:21.389346 19590 heartbeater.cc:464] Registering TS with master...
I20260809 23:15:21.389503 19590 heartbeater.cc:511] Master 127.2.76.254:44937 requested a full tablet report, sending...
I20260809 23:15:21.389925 19297 ts_manager.cc:194] Registered new tserver with Master: 2934b98199d14fce87fa5178d6dd542e (127.2.76.193:43595)
I20260809 23:15:21.390610 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 19390
I20260809 23:15:21.390702 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:15:21.390863 2355 external_mini_cluster.cc:1428] Setting key 45b9da07d35b4ae255abca60186a1143
I20260809 23:15:21.391816 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:44937
--builtin_ntp_servers=127.2.76.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 23:15:21.465368 19594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:21.465525 19594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:21.465544 19594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:21.466526 19594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:21.466557 19594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:21.466580 19594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:15:21.467703 19594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:44937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:21.467850 19594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:21.468048 19594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:21.472484 19603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:21.472544 19601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:21.472482 19605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:21.472576 19594 server_base.cc:1034] running on GCE node
I20260809 23:15:21.472860 19594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:21.473069 19594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:21.474198 19594 hybrid_clock.cc:648] HybridClock initialized: now 1786317321474185 us; error 32 us; skew 500 ppm
I20260809 23:15:21.475474 19594 webserver.cc:460] Webserver started at http://127.2.76.194:43189/ using document root <none> and password file <none>
I20260809 23:15:21.475677 19594 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:21.475741 19594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:21.475821 19594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:21.476770 19594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "ae45c944fa7f43b5bbfb8c801c86e4fd"
format_stamp: "Formatted at 2026-08-09 23:15:21 on dist-test-slave-7vjv"
server_key: "d5936b13d9e6b83abab01c3be022b2fc"
server_key_iv: "394578f2c2ac76bb8c3f3383116294e9"
server_key_version: "encryptionkey@0"
I20260809 23:15:21.477030 19594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "ae45c944fa7f43b5bbfb8c801c86e4fd"
format_stamp: "Formatted at 2026-08-09 23:15:21 on dist-test-slave-7vjv"
server_key: "d5936b13d9e6b83abab01c3be022b2fc"
server_key_iv: "394578f2c2ac76bb8c3f3383116294e9"
server_key_version: "encryptionkey@0"
I20260809 23:15:21.478132 19594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:15:21.478717 19613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.478896 19594 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:15:21.478978 19594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "ae45c944fa7f43b5bbfb8c801c86e4fd"
format_stamp: "Formatted at 2026-08-09 23:15:21 on dist-test-slave-7vjv"
server_key: "d5936b13d9e6b83abab01c3be022b2fc"
server_key_iv: "394578f2c2ac76bb8c3f3383116294e9"
server_key_version: "encryptionkey@0"
I20260809 23:15:21.479064 19594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:21.481875 19594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:21.482092 19594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:21.482213 19594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:21.482422 19594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:15:21.482679 19594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:21.482722 19594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.482755 19594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:21.482779 19594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.488232 19594 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:44461
I20260809 23:15:21.488291 19819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:44461 every 8 connection(s)
I20260809 23:15:21.488600 19594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:15:21.491006 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:21.489200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:50871 (local address 127.2.76.254:44937)
0809 23:15:21.489262 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:21.489265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:21.489490 (+ 225us) server_negotiation.cc:407] Connection header received
0809 23:15:21.489824 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:21.489825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:21.489830 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:21.489876 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:21.490817 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:21.490818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:21.490819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:15:21.490822 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:21.490833 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:15:21.490837 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:21.490879 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:21.490946 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 23:15:21.490957 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:15:21.491155 19823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:21.489204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:44937 (local address 127.2.76.194:50871)
0809 23:15:21.489398 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:21.489423 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:21.489754 (+ 331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:21.489937 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:21.489940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:21.489946 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:21.490702 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:15:21.490703 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:21.490725 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:15:21.490748 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:15:21.490886 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:15:21.490891 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:15:21.490893 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:15:21.490944 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 23:15:21.490986 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
I20260809 23:15:21.491544 19820 heartbeater.cc:347] Connected to a master server at 127.2.76.254:44937
I20260809 23:15:21.491618 19820 heartbeater.cc:464] Registering TS with master...
I20260809 23:15:21.491711 19820 heartbeater.cc:511] Master 127.2.76.254:44937 requested a full tablet report, sending...
I20260809 23:15:21.491983 19297 ts_manager.cc:194] Registered new tserver with Master: ae45c944fa7f43b5bbfb8c801c86e4fd (127.2.76.194:44461)
I20260809 23:15:21.496366 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 19594
I20260809 23:15:21.496438 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:15:21.496707 2355 external_mini_cluster.cc:1428] Setting key ffb94139f3cc9210909a3611ca0898d6
I20260809 23:15:21.497502 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:44937
--builtin_ntp_servers=127.2.76.212:37157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 23:15:21.570693 19824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:21.570892 19824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:21.570925 19824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:21.571897 19824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:21.571933 19824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:21.571954 19824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:15:21.573184 19824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:37157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:44937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:21.573411 19824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:21.573666 19824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:21.578327 19834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:21.578464 19824 server_base.cc:1034] running on GCE node
W20260809 23:15:21.578312 19840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:21.578358 19835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:21.578789 19824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:21.578975 19824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:21.580143 19824 hybrid_clock.cc:648] HybridClock initialized: now 1786317321580120 us; error 36 us; skew 500 ppm
I20260809 23:15:21.581326 19824 webserver.cc:460] Webserver started at http://127.2.76.195:39303/ using document root <none> and password file <none>
I20260809 23:15:21.581532 19824 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:21.581573 19824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:21.581648 19824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:21.582665 19824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "d5ca54635f544f0ea28fa433d0e3a108"
format_stamp: "Formatted at 2026-08-09 23:15:21 on dist-test-slave-7vjv"
server_key: "0ac2fa912b0dede4118e16ff9ba10b0a"
server_key_iv: "9bed795fc2c91680352a3f38f401a9c1"
server_key_version: "encryptionkey@0"
I20260809 23:15:21.582952 19824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "d5ca54635f544f0ea28fa433d0e3a108"
format_stamp: "Formatted at 2026-08-09 23:15:21 on dist-test-slave-7vjv"
server_key: "0ac2fa912b0dede4118e16ff9ba10b0a"
server_key_iv: "9bed795fc2c91680352a3f38f401a9c1"
server_key_version: "encryptionkey@0"
I20260809 23:15:21.584187 19824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:15:21.584887 19851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.585089 19824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:15:21.585170 19824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "d5ca54635f544f0ea28fa433d0e3a108"
format_stamp: "Formatted at 2026-08-09 23:15:21 on dist-test-slave-7vjv"
server_key: "0ac2fa912b0dede4118e16ff9ba10b0a"
server_key_iv: "9bed795fc2c91680352a3f38f401a9c1"
server_key_version: "encryptionkey@0"
I20260809 23:15:21.585244 19824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:21.587968 19824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:21.588215 19824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:21.588326 19824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:21.588507 19824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:15:21.588757 19824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:21.588785 19824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.588825 19824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:21.588850 19824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:21.594036 19824 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:38495
I20260809 23:15:21.594094 20027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:38495 every 8 connection(s)
I20260809 23:15:21.594372 19824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:15:21.596792 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:21.594962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:42607 (local address 127.2.76.254:44937)
0809 23:15:21.595006 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:21.595008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:21.595203 (+ 195us) server_negotiation.cc:407] Connection header received
0809 23:15:21.595561 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:21.595562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:21.595566 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:21.595609 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:21.596628 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:21.596629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:21.596629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:15:21.596632 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:21.596641 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:15:21.596643 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:21.596684 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:21.596736 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 23:15:21.596747 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 23:15:21.596976 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:21.594928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:44937 (local address 127.2.76.195:42607)
0809 23:15:21.595094 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:21.595126 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:21.595492 (+ 366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:21.595659 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:21.595661 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:21.595668 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:21.596517 (+ 849us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:15:21.596518 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:21.596539 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:15:21.596563 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:15:21.596685 (+ 122us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:15:21.596687 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:15:21.596689 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:15:21.596736 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 23:15:21.596774 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260809 23:15:21.597421 20030 heartbeater.cc:347] Connected to a master server at 127.2.76.254:44937
I20260809 23:15:21.597523 20030 heartbeater.cc:464] Registering TS with master...
I20260809 23:15:21.597679 20030 heartbeater.cc:511] Master 127.2.76.254:44937 requested a full tablet report, sending...
I20260809 23:15:21.598001 19297 ts_manager.cc:194] Registered new tserver with Master: d5ca54635f544f0ea28fa433d0e3a108 (127.2.76.195:38495)
I20260809 23:15:21.602392 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 19824
I20260809 23:15:21.602507 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:15:21.602692 2355 external_mini_cluster.cc:1428] Setting key 20e8d0bb0127c7ce3ba43cd5b18b2120
I20260809 23:15:21.603367 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 23:15:21.605104 19352 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:21.604656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40264 (local address 127.2.76.254:44937)
0809 23:15:21.604719 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:21.604721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:21.604783 (+ 62us) server_negotiation.cc:407] Connection header received
0809 23:15:21.604850 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:21.604850 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:21.604855 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:21.604898 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:21.605046 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40264: BlockingRecv error: recv got EOF from 127.0.0.1:40264 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:15:21.605156 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 19390
I20260809 23:15:21.628351 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 19594
I20260809 23:15:21.641011 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 19824
I20260809 23:15:21.654072 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 19241
2026-08-09T23:15:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (523 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:15:21 dist-test-slave-7vjv krb5kdc[20050](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:15:21 dist-test-slave-7vjv krb5kdc[20050](info): set up 2 sockets
Aug 09 23:15:21 dist-test-slave-7vjv krb5kdc[20050](info): commencing operation
krb5kdc: starting...
W20260809 23:15:22.794209 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:15:22 dist-test-slave-7vjv krb5kdc[20050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:15:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:15:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:15:22.897017 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:36501
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:36501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:15:22.968384 20075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:22.968585 20075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:22.968619 20075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:22.969588 20075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:15:22.969645 20075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:22.969667 20075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:22.969686 20075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:15:22.969712 20075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:15:22.971057 20075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:36501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:36501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:22.971315 20075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:22.971514 20075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:15:22.976117 20075 server_base.cc:1034] running on GCE node
W20260809 23:15:22.976079 20082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:22.976179 20088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:22.976079 20084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:22.976615 20075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:22.976874 20075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:22.978015 20075 hybrid_clock.cc:648] HybridClock initialized: now 1786317322977999 us; error 47 us; skew 500 ppm
Aug 09 23:15:22 dist-test-slave-7vjv krb5kdc[20050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317322, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:15:22.980598 20075 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:15:22.980965 20075 webserver.cc:460] Webserver started at http://127.2.76.254:35901/ using document root <none> and password file <none>
I20260809 23:15:22.981155 20075 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:22.981190 20075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:22.981266 20075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:22.982131 20075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "4f1174dcd23b41629187a557c834fe04"
format_stamp: "Formatted at 2026-08-09 23:15:22 on dist-test-slave-7vjv"
server_key: "3812185200e665e54962d65d7e9fcac6"
server_key_iv: "5c27f953ce8dea027deafbf77ce9679b"
server_key_version: "encryptionkey@0"
I20260809 23:15:22.982419 20075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "4f1174dcd23b41629187a557c834fe04"
format_stamp: "Formatted at 2026-08-09 23:15:22 on dist-test-slave-7vjv"
server_key: "3812185200e665e54962d65d7e9fcac6"
server_key_iv: "5c27f953ce8dea027deafbf77ce9679b"
server_key_version: "encryptionkey@0"
I20260809 23:15:22.983812 20075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:15:22.984513 20101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:22.984685 20075 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:22.984786 20075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "4f1174dcd23b41629187a557c834fe04"
format_stamp: "Formatted at 2026-08-09 23:15:22 on dist-test-slave-7vjv"
server_key: "3812185200e665e54962d65d7e9fcac6"
server_key_iv: "5c27f953ce8dea027deafbf77ce9679b"
server_key_version: "encryptionkey@0"
I20260809 23:15:22.984855 20075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:22.999053 20075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:23.000623 20075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:23.000772 20075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:23.004626 20075 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:36501
I20260809 23:15:23.004669 20182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:36501 every 8 connection(s)
I20260809 23:15:23.004951 20075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:15:23.005738 20183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:23.007712 20183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04: Bootstrap starting.
I20260809 23:15:23.008309 20183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:23.008594 20183 log.cc:824] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04: Log is configured to *not* fsync() on all Append() calls
I20260809 23:15:23.009243 20183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04: No bootstrap required, opened a new log
I20260809 23:15:23.010695 20183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f1174dcd23b41629187a557c834fe04" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36501 } }
I20260809 23:15:23.010891 20183 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:23.010902 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 20075
I20260809 23:15:23.010927 20183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1174dcd23b41629187a557c834fe04, State: Initialized, Role: FOLLOWER
I20260809 23:15:23.010991 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:15:23.011096 20183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f1174dcd23b41629187a557c834fe04" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36501 } }
I20260809 23:15:23.011176 2355 external_mini_cluster.cc:1428] Setting key 123832782acc4fcf6348fc7754b5e0ec
I20260809 23:15:23.011178 20183 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:15:23.011211 20183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:15:23.011261 20183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:23.011942 20183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f1174dcd23b41629187a557c834fe04" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36501 } }
I20260809 23:15:23.012061 20183 leader_election.cc:302] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f1174dcd23b41629187a557c834fe04; no voters:
I20260809 23:15:23.012223 20183 leader_election.cc:288] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:15:23.012275 20193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:15:23.012454 20183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:15:23.012452 20193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [term 1 LEADER]: Becoming Leader. State: Replica: 4f1174dcd23b41629187a557c834fe04, State: Running, Role: LEADER
I20260809 23:15:23.012575 20193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f1174dcd23b41629187a557c834fe04" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36501 } }
I20260809 23:15:23.012912 20193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f1174dcd23b41629187a557c834fe04. Latest consensus state: current_term: 1 leader_uuid: "4f1174dcd23b41629187a557c834fe04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f1174dcd23b41629187a557c834fe04" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36501 } } }
I20260809 23:15:23.012972 20193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [sys.catalog]: This master's current role is: LEADER
I20260809 23:15:23.013252 20195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f1174dcd23b41629187a557c834fe04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f1174dcd23b41629187a557c834fe04" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36501 } } }
I20260809 23:15:23.013374 20195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04 [sys.catalog]: This master's current role is: LEADER
I20260809 23:15:23.013515 20207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:15:23.014025 20207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:15:23.015689 20207 catalog_manager.cc:1347] Generated new cluster ID: 3b5227dfb1874460b89b32367f5ea377
I20260809 23:15:23.015740 20207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:15:23 dist-test-slave-7vjv krb5kdc[20050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:15:23.022308 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.011620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59944 (local address 127.2.76.254:36501)
0809 23:15:23.012046 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:23.012054 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:23.012084 (+ 30us) server_negotiation.cc:407] Connection header received
0809 23:15:23.012202 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:23.012207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:23.012274 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:23.013150 (+ 876us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:23.013237 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.014759 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.015641 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.016199 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.016294 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.018853 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:23.018856 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:23.018857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:15:23.018866 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:23.020724 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:23.021116 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:23.021119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:23.021119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:23.021232 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:23.021436 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:23.021437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:23.021437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:23.021683 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:23.021789 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:23.021927 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 23:15:23.022039 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":356,"threads_started":1}
I20260809 23:15:23.031570 20207 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:15:23.032213 20207 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:15:23.035355 20207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04: Generated new TSK 0
I20260809 23:15:23.035516 20207 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:15:23.062742 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36501
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:15:23.135577 20224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:23.135772 20224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:23.135805 20224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:23.136816 20224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:23.136869 20224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:23.136904 20224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:15:23.138131 20224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:23.138356 20224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:23.138548 20224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:23.142874 20238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:23.142951 20235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:23.142904 20234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:23.143034 20224 server_base.cc:1034] running on GCE node
I20260809 23:15:23.143383 20224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:23.143627 20224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:23.144778 20224 hybrid_clock.cc:648] HybridClock initialized: now 1786317323144781 us; error 38 us; skew 500 ppm
Aug 09 23:15:23 dist-test-slave-7vjv krb5kdc[20050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317323, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:15:23.147043 20224 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:15:23.147434 20224 webserver.cc:460] Webserver started at http://127.2.76.193:35029/ using document root <none> and password file <none>
I20260809 23:15:23.147630 20224 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:23.147668 20224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:23.147811 20224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:23.148710 20224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "360a5512821f4a20917b07fac6ddbeca"
format_stamp: "Formatted at 2026-08-09 23:15:23 on dist-test-slave-7vjv"
server_key: "286eb4ba9592da511288138b4b470ac8"
server_key_iv: "c5ee2f4c4dcea400444f90170f184235"
server_key_version: "encryptionkey@0"
I20260809 23:15:23.149034 20224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "360a5512821f4a20917b07fac6ddbeca"
format_stamp: "Formatted at 2026-08-09 23:15:23 on dist-test-slave-7vjv"
server_key: "286eb4ba9592da511288138b4b470ac8"
server_key_iv: "c5ee2f4c4dcea400444f90170f184235"
server_key_version: "encryptionkey@0"
I20260809 23:15:23.150235 20224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:15:23.150872 20250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:23.151053 20224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:15:23.151134 20224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "360a5512821f4a20917b07fac6ddbeca"
format_stamp: "Formatted at 2026-08-09 23:15:23 on dist-test-slave-7vjv"
server_key: "286eb4ba9592da511288138b4b470ac8"
server_key_iv: "c5ee2f4c4dcea400444f90170f184235"
server_key_version: "encryptionkey@0"
I20260809 23:15:23.151181 20224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:23.172374 20224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:23.174234 20224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:23.174388 20224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:23.174572 20224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:15:23.174839 20224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:23.174922 20224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:23.174950 20224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:23.174975 20224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:23.180037 20224 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:33983
I20260809 23:15:23.180068 20430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:33983 every 8 connection(s)
I20260809 23:15:23.180356 20224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:15:23 dist-test-slave-7vjv krb5kdc[20050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317323, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:15:23.187433 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 20224
I20260809 23:15:23.187552 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:15:23.187726 2355 external_mini_cluster.cc:1428] Setting key 02449e90bfb8f07b38a239a1616d20e2
I20260809 23:15:23.188082 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.181020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:36627 (local address 127.2.76.254:36501)
0809 23:15:23.181058 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:23.181060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:23.181254 (+ 194us) server_negotiation.cc:407] Connection header received
0809 23:15:23.181670 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:23.181671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:23.181681 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:23.181735 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:23.182518 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.182933 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.183722 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.183884 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.183926 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.185722 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:23.185723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:23.185723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:15:23.185729 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:23.186724 (+ 995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:23.187061 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:23.187061 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:23.187062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:23.187126 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:23.187361 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:23.187361 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:23.187362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:23.187449 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:23.187524 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:23.187890 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 23:15:23.187978 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:15:23.188225 20436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.180974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:36501 (local address 127.2.76.193:36627)
0809 23:15:23.181135 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:23.181170 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:23.181594 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:23.181791 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:23.181794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:23.182277 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:23.182431 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.182435 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.183006 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.183007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.183657 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.183658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.183949 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.183950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.184032 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.184486 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:23.184503 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:23.185644 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:15:23.186801 (+ 1157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:23.186809 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:23.186811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:23.187002 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:23.187188 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:23.187189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:23.187189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:23.187301 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:23.187568 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:15:23.187575 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:15:23.187713 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 23:15:23.187850 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 23:15:23.188021 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260809 23:15:23.188653 20431 heartbeater.cc:347] Connected to a master server at 127.2.76.254:36501
I20260809 23:15:23.188738 20431 heartbeater.cc:464] Registering TS with master...
I20260809 23:15:23.188889 20431 heartbeater.cc:511] Master 127.2.76.254:36501 requested a full tablet report, sending...
I20260809 23:15:23.189356 20128 ts_manager.cc:194] Registered new tserver with Master: 360a5512821f4a20917b07fac6ddbeca (127.2.76.193:33983)
I20260809 23:15:23.190117 20128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:36627
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:15:23.210556 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36501
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:15:23.284017 20441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:23.284255 20441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:23.284293 20441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:23.285331 20441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:23.285389 20441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:23.285426 20441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:15:23.286659 20441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:23.286892 20441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:23.287109 20441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:23.291348 20450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:23.291356 20454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:23.291391 20449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:23.291541 20441 server_base.cc:1034] running on GCE node
I20260809 23:15:23.291724 20441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:23.291949 20441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:23.293104 20441 hybrid_clock.cc:648] HybridClock initialized: now 1786317323293103 us; error 38 us; skew 500 ppm
Aug 09 23:15:23 dist-test-slave-7vjv krb5kdc[20050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317323, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:15:23.295189 20441 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:15:23.295507 20441 webserver.cc:460] Webserver started at http://127.2.76.194:43679/ using document root <none> and password file <none>
I20260809 23:15:23.295657 20441 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:23.295691 20441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:23.295761 20441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:23.296667 20441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "4c00e546255f4456a481e1f224b52b71"
format_stamp: "Formatted at 2026-08-09 23:15:23 on dist-test-slave-7vjv"
server_key: "618fa2537c042754cd0a2aa2a5783629"
server_key_iv: "0da96477298cad43047b151d46307b1d"
server_key_version: "encryptionkey@0"
I20260809 23:15:23.296952 20441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "4c00e546255f4456a481e1f224b52b71"
format_stamp: "Formatted at 2026-08-09 23:15:23 on dist-test-slave-7vjv"
server_key: "618fa2537c042754cd0a2aa2a5783629"
server_key_iv: "0da96477298cad43047b151d46307b1d"
server_key_version: "encryptionkey@0"
I20260809 23:15:23.298171 20441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:15:23.298802 20463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:23.298992 20441 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:15:23.299078 20441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "4c00e546255f4456a481e1f224b52b71"
format_stamp: "Formatted at 2026-08-09 23:15:23 on dist-test-slave-7vjv"
server_key: "618fa2537c042754cd0a2aa2a5783629"
server_key_iv: "0da96477298cad43047b151d46307b1d"
server_key_version: "encryptionkey@0"
I20260809 23:15:23.299137 20441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:23.314165 20441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:23.316156 20441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:23.316253 20441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:23.316406 20441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:15:23.316620 20441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:23.316653 20441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:23.316670 20441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:23.316681 20441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:23.321941 20441 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:38023
I20260809 23:15:23.322011 20644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:38023 every 8 connection(s)
I20260809 23:15:23.322297 20441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:15:23.325007 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 20441
I20260809 23:15:23.325109 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:15:23.325277 2355 external_mini_cluster.cc:1428] Setting key 4ba58879562e0d7ee72000888f521c03
Aug 09 23:15:23 dist-test-slave-7vjv krb5kdc[20050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317323, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:15:23.331027 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.322927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:55853 (local address 127.2.76.254:36501)
0809 23:15:23.323002 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:23.323004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:23.323170 (+ 166us) server_negotiation.cc:407] Connection header received
0809 23:15:23.323599 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:23.323600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:23.323612 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:23.323662 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:23.324439 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.324898 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.326008 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.326195 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.326261 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.328203 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:23.328204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:23.328204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:15:23.328211 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:23.329664 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:23.330041 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:23.330042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:23.330042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:23.330120 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:23.330399 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:23.330400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:23.330400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:23.330550 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:23.330621 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:23.330843 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 23:15:23.330922 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 23:15:23.331284 20651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.322892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:36501 (local address 127.2.76.194:55853)
0809 23:15:23.323057 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:23.323098 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:23.323505 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:23.323716 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:23.323719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:23.324200 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:23.324350 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.324354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.324982 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.324983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.325939 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.325941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.326269 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.326270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.326387 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.326797 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:23.326804 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:23.328072 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:15:23.329748 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:23.329754 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:23.329755 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:23.329953 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:23.330186 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:23.330187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:23.330187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:23.330331 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:23.330616 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:15:23.330621 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:15:23.330716 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 23:15:23.330815 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 23:15:23.331014 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260809 23:15:23.331755 20647 heartbeater.cc:347] Connected to a master server at 127.2.76.254:36501
I20260809 23:15:23.331974 20647 heartbeater.cc:464] Registering TS with master...
I20260809 23:15:23.332182 20647 heartbeater.cc:511] Master 127.2.76.254:36501 requested a full tablet report, sending...
I20260809 23:15:23.332516 20128 ts_manager.cc:194] Registered new tserver with Master: 4c00e546255f4456a481e1f224b52b71 (127.2.76.194:38023)
I20260809 23:15:23.332866 20128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:55853
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:15:23.348305 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36501
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:39185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:15:23.419170 20658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:15:23.419325 20658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:15:23.419343 20658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:15:23.420416 20658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:15:23.420450 20658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:15:23.420493 20658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:15:23.421866 20658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:39185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:36501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:15:23.422019 20658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:15:23.422168 20658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:15:23.426458 20666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:23.426453 20677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:15:23.426455 20667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:15:23.426739 20658 server_base.cc:1034] running on GCE node
I20260809 23:15:23.426905 20658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:15:23.427086 20658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:15:23.428221 20658 hybrid_clock.cc:648] HybridClock initialized: now 1786317323428211 us; error 27 us; skew 500 ppm
Aug 09 23:15:23 dist-test-slave-7vjv krb5kdc[20050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317323, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:15:23.430605 20658 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:15:23.431046 20658 webserver.cc:460] Webserver started at http://127.2.76.195:38143/ using document root <none> and password file <none>
I20260809 23:15:23.431291 20658 fs_manager.cc:359] Metadata directory not provided
I20260809 23:15:23.431354 20658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:15:23.431488 20658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:15:23.432509 20658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "4091e34c8fbe49ddab3c47d6a759b2fd"
format_stamp: "Formatted at 2026-08-09 23:15:23 on dist-test-slave-7vjv"
server_key: "309d1269ae0268d867a0b6a783250b77"
server_key_iv: "57a9d65a18653273672a484f6494e360"
server_key_version: "encryptionkey@0"
I20260809 23:15:23.432860 20658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "4091e34c8fbe49ddab3c47d6a759b2fd"
format_stamp: "Formatted at 2026-08-09 23:15:23 on dist-test-slave-7vjv"
server_key: "309d1269ae0268d867a0b6a783250b77"
server_key_iv: "57a9d65a18653273672a484f6494e360"
server_key_version: "encryptionkey@0"
I20260809 23:15:23.434132 20658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:15:23.434801 20688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:23.434965 20658 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:15:23.435068 20658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "4091e34c8fbe49ddab3c47d6a759b2fd"
format_stamp: "Formatted at 2026-08-09 23:15:23 on dist-test-slave-7vjv"
server_key: "309d1269ae0268d867a0b6a783250b77"
server_key_iv: "57a9d65a18653273672a484f6494e360"
server_key_version: "encryptionkey@0"
I20260809 23:15:23.435137 20658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:15:23.450702 20658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:15:23.452768 20658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:15:23.452908 20658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:15:23.453133 20658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:15:23.453421 20658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:15:23.453490 20658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:23.453521 20658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:15:23.453544 20658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:15:23.458906 20658 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:33883
I20260809 23:15:23.458963 20853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:33883 every 8 connection(s)
I20260809 23:15:23.459201 20658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:15:23.462110 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 20658
I20260809 23:15:23.462212 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:15:23.462384 2355 external_mini_cluster.cc:1428] Setting key 1ab73843842842f24d8a9c8da90f215d
Aug 09 23:15:23 dist-test-slave-7vjv krb5kdc[20050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317323, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:15:23.466904 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.459787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:55957 (local address 127.2.76.254:36501)
0809 23:15:23.459850 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:23.459851 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:23.460036 (+ 185us) server_negotiation.cc:407] Connection header received
0809 23:15:23.460503 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:23.460505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:23.460515 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:23.460563 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:23.461319 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.461731 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.462691 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.462976 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.463044 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.464896 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:23.464897 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:23.464897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:15:23.464902 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:23.465910 (+ 1008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:23.466214 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:23.466215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:23.466215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:23.466261 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:23.466463 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:23.466463 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:23.466463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:23.466553 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:23.466598 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:23.466793 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 23:15:23.466848 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:15:23.467053 20857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.459750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:36501 (local address 127.2.76.195:55957)
0809 23:15:23.459912 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:23.459948 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:23.460421 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:23.460605 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:23.460608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:23.461087 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:15:23.461252 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.461256 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.461798 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.461798 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.462620 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.462622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.463066 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.463067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.463184 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.463611 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:15:23.463623 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:15:23.464820 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:15:23.465970 (+ 1150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:23.465976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:23.465977 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:23.466157 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:23.466309 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:15:23.466310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:15:23.466310 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:15:23.466415 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:15:23.466600 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:15:23.466604 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:15:23.466695 (+ 91us) client_negotiation.cc:769] Sending connection context
0809 23:15:23.466766 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 23:15:23.466863 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260809 23:15:23.467468 20854 heartbeater.cc:347] Connected to a master server at 127.2.76.254:36501
I20260809 23:15:23.467540 20854 heartbeater.cc:464] Registering TS with master...
I20260809 23:15:23.467697 20854 heartbeater.cc:511] Master 127.2.76.254:36501 requested a full tablet report, sending...
I20260809 23:15:23.468036 20128 ts_manager.cc:194] Registered new tserver with Master: 4091e34c8fbe49ddab3c47d6a759b2fd (127.2.76.195:33883)
I20260809 23:15:23.468386 20128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:55957
I20260809 23:15:23.473853 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:15:23.480020 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.475050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59954 (local address 127.2.76.254:36501)
0809 23:15:23.475110 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:23.475112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:23.475189 (+ 77us) server_negotiation.cc:407] Connection header received
0809 23:15:23.475223 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:23.475224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:23.475231 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:23.475272 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:15:23.475747 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.476193 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.476818 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.476984 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.477033 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.477943 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:15:23.477944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:15:23.477944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:15:23.477950 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:15:23.479209 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:23.479465 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:23.479466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:23.479466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:23.479507 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:15:23.479646 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:15:23.479646 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:15:23.479646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:15:23.479751 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:15:23.479815 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:23.479898 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 23:15:23.479947 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 23:15:23.481328 20128 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:15:23.482683 20128 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:15:23.488529 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.484619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:33983 (local address 127.0.0.1:37130)
0809 23:15:23.484771 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:23.484805 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:23.484865 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:23.485147 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:23.485149 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:23.485151 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:15:23.485207 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.485208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.486215 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.486216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.487321 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.487324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.488247 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.488248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.488321 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.488333 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 23:15:23.488403 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 23:15:23.488453 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":63,"threads_started":1}
I20260809 23:15:23.488612 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.484810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37130 (local address 127.2.76.193:33983)
0809 23:15:23.485000 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:23.485005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:23.485015 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:15:23.485053 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:23.485055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:23.485069 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:23.485133 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:15:23.485279 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.486099 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.487414 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.488076 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.488161 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.488212 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:23.488449 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 23:15:23.488490 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":80,"threads_started":1}
I20260809 23:15:23.488754 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.484649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:33883 (local address 127.0.0.1:59992)
0809 23:15:23.484988 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:23.484997 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:23.485065 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:23.485302 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:23.485303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:23.485305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:15:23.485359 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.485361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.486589 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.486590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.487670 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.487672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.488513 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.488514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.488594 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.488598 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:15:23.488651 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 23:15:23.488687 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":29,"thread_start_us":83,"threads_started":1}
I20260809 23:15:23.488986 20880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.484831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59992 (local address 127.2.76.195:33883)
0809 23:15:23.484973 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:23.484978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:23.485054 (+ 76us) server_negotiation.cc:407] Connection header received
0809 23:15:23.485130 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:23.485132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:23.485147 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:23.485220 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:15:23.485442 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.486480 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.487751 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.488373 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.488425 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.488469 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:23.488864 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 23:15:23.488910 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
I20260809 23:15:23.489336 20872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.484619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:38023 (local address 127.0.0.1:60548)
0809 23:15:23.484957 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:23.484966 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:23.485016 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:23.485504 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:23.485504 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:23.485506 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:15:23.485539 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.485541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.486704 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.486705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.487633 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.487635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.489112 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.489113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.489179 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.489182 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:15:23.489245 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 23:15:23.489281 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":2,"thread_start_us":57,"threads_started":1}
I20260809 23:15:23.489437 20332 tablet_service.cc:1467] Processing CreateTablet for tablet 3871387002fc436f813a1cb0235b5e0d (DEFAULT_TABLE table=test-table [id=14642944bcc14a3f91e0eb0bae142d71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:15:23.489590 20750 tablet_service.cc:1467] Processing CreateTablet for tablet 3871387002fc436f813a1cb0235b5e0d (DEFAULT_TABLE table=test-table [id=14642944bcc14a3f91e0eb0bae142d71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:15:23.489822 20332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3871387002fc436f813a1cb0235b5e0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:23.489867 20750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3871387002fc436f813a1cb0235b5e0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:23.492923 20882 tablet_bootstrap.cc:492] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca: Bootstrap starting.
I20260809 23:15:23.493000 20884 tablet_bootstrap.cc:492] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd: Bootstrap starting.
I20260809 23:15:23.493148 20881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.484814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60548 (local address 127.2.76.194:38023)
0809 23:15:23.485194 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:23.485199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:23.485207 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:15:23.485261 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:23.485263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:23.485345 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:23.485421 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:15:23.485619 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.486626 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.487724 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.488538 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.489102 (+ 564us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.489167 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:23.493038 (+ 3871us) server_negotiation.cc:299] Negotiation successful
0809 23:15:23.493079 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":80,"threads_started":1}
I20260809 23:15:23.493644 20884 tablet_bootstrap.cc:654] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:23.493856 20882 tablet_bootstrap.cc:654] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:23.493801 20543 tablet_service.cc:1467] Processing CreateTablet for tablet 3871387002fc436f813a1cb0235b5e0d (DEFAULT_TABLE table=test-table [id=14642944bcc14a3f91e0eb0bae142d71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:15:23.494153 20884 log.cc:824] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd: Log is configured to *not* fsync() on all Append() calls
I20260809 23:15:23.494150 20543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3871387002fc436f813a1cb0235b5e0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:15:23.494354 20882 log.cc:824] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca: Log is configured to *not* fsync() on all Append() calls
I20260809 23:15:23.494886 20884 tablet_bootstrap.cc:492] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd: No bootstrap required, opened a new log
I20260809 23:15:23.494953 20884 ts_tablet_manager.cc:1397] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:15:23.495127 20882 tablet_bootstrap.cc:492] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca: No bootstrap required, opened a new log
I20260809 23:15:23.495186 20882 ts_tablet_manager.cc:1397] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:15:23.496714 20884 raft_consensus.cc:348] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c00e546255f4456a481e1f224b52b71" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 38023 } } peers { permanent_uuid: "360a5512821f4a20917b07fac6ddbeca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33983 } } peers { permanent_uuid: "4091e34c8fbe49ddab3c47d6a759b2fd" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33883 } }
I20260809 23:15:23.496866 20884 raft_consensus.cc:374] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:23.496905 20884 raft_consensus.cc:729] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4091e34c8fbe49ddab3c47d6a759b2fd, State: Initialized, Role: FOLLOWER
I20260809 23:15:23.496915 20882 raft_consensus.cc:348] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c00e546255f4456a481e1f224b52b71" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 38023 } } peers { permanent_uuid: "360a5512821f4a20917b07fac6ddbeca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33983 } } peers { permanent_uuid: "4091e34c8fbe49ddab3c47d6a759b2fd" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33883 } }
I20260809 23:15:23.497022 20884 consensus_queue.cc:260] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c00e546255f4456a481e1f224b52b71" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 38023 } } peers { permanent_uuid: "360a5512821f4a20917b07fac6ddbeca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33983 } } peers { permanent_uuid: "4091e34c8fbe49ddab3c47d6a759b2fd" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33883 } }
I20260809 23:15:23.497068 20882 raft_consensus.cc:374] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:23.497102 20882 raft_consensus.cc:729] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360a5512821f4a20917b07fac6ddbeca, State: Initialized, Role: FOLLOWER
I20260809 23:15:23.497114 20887 tablet_bootstrap.cc:492] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71: Bootstrap starting.
I20260809 23:15:23.497200 20882 consensus_queue.cc:260] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c00e546255f4456a481e1f224b52b71" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 38023 } } peers { permanent_uuid: "360a5512821f4a20917b07fac6ddbeca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33983 } } peers { permanent_uuid: "4091e34c8fbe49ddab3c47d6a759b2fd" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33883 } }
I20260809 23:15:23.497268 20884 ts_tablet_manager.cc:1428] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:15:23.497376 20854 heartbeater.cc:503] Master 127.2.76.254:36501 was elected leader, sending a full tablet report...
I20260809 23:15:23.497433 20882 ts_tablet_manager.cc:1428] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:15:23.497555 20431 heartbeater.cc:503] Master 127.2.76.254:36501 was elected leader, sending a full tablet report...
I20260809 23:15:23.497725 20887 tablet_bootstrap.cc:654] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71: Neither blocks nor log segments found. Creating new log.
I20260809 23:15:23.498220 20887 log.cc:824] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71: Log is configured to *not* fsync() on all Append() calls
I20260809 23:15:23.498971 20887 tablet_bootstrap.cc:492] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71: No bootstrap required, opened a new log
I20260809 23:15:23.499025 20887 ts_tablet_manager.cc:1397] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:15:23.500283 20887 raft_consensus.cc:348] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c00e546255f4456a481e1f224b52b71" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 38023 } } peers { permanent_uuid: "360a5512821f4a20917b07fac6ddbeca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33983 } } peers { permanent_uuid: "4091e34c8fbe49ddab3c47d6a759b2fd" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33883 } }
I20260809 23:15:23.500386 20887 raft_consensus.cc:374] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:15:23.500404 20887 raft_consensus.cc:729] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c00e546255f4456a481e1f224b52b71, State: Initialized, Role: FOLLOWER
I20260809 23:15:23.500501 20887 consensus_queue.cc:260] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c00e546255f4456a481e1f224b52b71" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 38023 } } peers { permanent_uuid: "360a5512821f4a20917b07fac6ddbeca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33983 } } peers { permanent_uuid: "4091e34c8fbe49ddab3c47d6a759b2fd" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33883 } }
I20260809 23:15:23.500696 20887 ts_tablet_manager.cc:1428] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:15:23.500759 20647 heartbeater.cc:503] Master 127.2.76.254:36501 was elected leader, sending a full tablet report...
I20260809 23:15:23.533250 20888 raft_consensus.cc:482] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:15:23.533325 20888 raft_consensus.cc:504] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c00e546255f4456a481e1f224b52b71" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 38023 } } peers { permanent_uuid: "360a5512821f4a20917b07fac6ddbeca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33983 } } peers { permanent_uuid: "4091e34c8fbe49ddab3c47d6a759b2fd" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33883 } }
I20260809 23:15:23.533630 20888 leader_election.cc:288] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c00e546255f4456a481e1f224b52b71 (127.2.76.194:38023), 360a5512821f4a20917b07fac6ddbeca (127.2.76.193:33983)
I20260809 23:15:23.537014 20881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.533891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:48553 (local address 127.2.76.194:38023)
0809 23:15:23.533933 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:23.533934 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:23.533943 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:15:23.533965 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:23.533966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:23.533976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:23.534040 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:15:23.534186 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.534921 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.536197 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.536689 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.536760 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.536767 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:23.536923 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 23:15:23.536951 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:15:23.536999 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.533881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:45051 (local address 127.2.76.193:33983)
0809 23:15:23.533919 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:23.533921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:23.534095 (+ 174us) server_negotiation.cc:407] Connection header received
0809 23:15:23.534127 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:23.534127 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:23.534135 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:23.534197 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:15:23.534356 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.535004 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.536197 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.536689 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.536760 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.536767 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:23.536910 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 23:15:23.536943 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:15:23.537026 20857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.533795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:38023 (local address 127.2.76.195:48553)
0809 23:15:23.533848 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:23.533860 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:23.533906 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:23.534061 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:23.534062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:23.534063 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:15:23.534120 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.534121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.534994 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.534995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.536120 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.536122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.536767 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.536768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.536838 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.536843 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:15:23.536893 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 23:15:23.536934 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":8}
I20260809 23:15:23.537282 20893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.533795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:33983 (local address 127.2.76.195:45051)
0809 23:15:23.533997 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:23.534015 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:23.534082 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:23.534223 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:15:23.534223 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:15:23.534225 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:15:23.534287 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.534288 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.535080 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.535080 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.536119 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:15:23.536122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.536767 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.536768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:15:23.536838 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.536844 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 23:15:23.536893 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 23:15:23.536934 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":85,"thread_start_us":84,"threads_started":1}
I20260809 23:15:23.537437 20576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3871387002fc436f813a1cb0235b5e0d" candidate_uuid: "4091e34c8fbe49ddab3c47d6a759b2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c00e546255f4456a481e1f224b52b71" is_pre_election: true
I20260809 23:15:23.537523 20365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3871387002fc436f813a1cb0235b5e0d" candidate_uuid: "4091e34c8fbe49ddab3c47d6a759b2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360a5512821f4a20917b07fac6ddbeca" is_pre_election: true
I20260809 23:15:23.537580 20576 raft_consensus.cc:2454] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4091e34c8fbe49ddab3c47d6a759b2fd in term 0.
I20260809 23:15:23.537626 20365 raft_consensus.cc:2454] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4091e34c8fbe49ddab3c47d6a759b2fd in term 0.
I20260809 23:15:23.537791 20691 leader_election.cc:302] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4091e34c8fbe49ddab3c47d6a759b2fd, 4c00e546255f4456a481e1f224b52b71; no voters:
I20260809 23:15:23.537914 20888 raft_consensus.cc:2781] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:15:23.537964 20888 raft_consensus.cc:482] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:15:23.537983 20888 raft_consensus.cc:3037] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:23.538539 20888 raft_consensus.cc:504] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c00e546255f4456a481e1f224b52b71" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 38023 } } peers { permanent_uuid: "360a5512821f4a20917b07fac6ddbeca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33983 } } peers { permanent_uuid: "4091e34c8fbe49ddab3c47d6a759b2fd" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33883 } }
I20260809 23:15:23.538652 20888 leader_election.cc:288] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [CANDIDATE]: Term 1 election: Requested vote from peers 4c00e546255f4456a481e1f224b52b71 (127.2.76.194:38023), 360a5512821f4a20917b07fac6ddbeca (127.2.76.193:33983)
I20260809 23:15:23.538820 20365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3871387002fc436f813a1cb0235b5e0d" candidate_uuid: "4091e34c8fbe49ddab3c47d6a759b2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360a5512821f4a20917b07fac6ddbeca"
I20260809 23:15:23.538808 20576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3871387002fc436f813a1cb0235b5e0d" candidate_uuid: "4091e34c8fbe49ddab3c47d6a759b2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c00e546255f4456a481e1f224b52b71"
I20260809 23:15:23.538878 20365 raft_consensus.cc:3037] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:23.538904 20576 raft_consensus.cc:3037] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:15:23.539451 20365 raft_consensus.cc:2454] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4091e34c8fbe49ddab3c47d6a759b2fd in term 1.
I20260809 23:15:23.539515 20576 raft_consensus.cc:2454] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4091e34c8fbe49ddab3c47d6a759b2fd in term 1.
I20260809 23:15:23.539662 20694 leader_election.cc:302] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 360a5512821f4a20917b07fac6ddbeca, 4091e34c8fbe49ddab3c47d6a759b2fd; no voters:
I20260809 23:15:23.539773 20888 raft_consensus.cc:2781] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:15:23.539921 20888 raft_consensus.cc:686] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [term 1 LEADER]: Becoming Leader. State: Replica: 4091e34c8fbe49ddab3c47d6a759b2fd, State: Running, Role: LEADER
I20260809 23:15:23.540004 20888 consensus_queue.cc:237] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c00e546255f4456a481e1f224b52b71" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 38023 } } peers { permanent_uuid: "360a5512821f4a20917b07fac6ddbeca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33983 } } peers { permanent_uuid: "4091e34c8fbe49ddab3c47d6a759b2fd" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33883 } }
I20260809 23:15:23.540709 20128 catalog_manager.cc:5495] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 4091e34c8fbe49ddab3c47d6a759b2fd (127.2.76.195). New cstate: current_term: 1 leader_uuid: "4091e34c8fbe49ddab3c47d6a759b2fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c00e546255f4456a481e1f224b52b71" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 38023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "360a5512821f4a20917b07fac6ddbeca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4091e34c8fbe49ddab3c47d6a759b2fd" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33883 } health_report { overall_health: HEALTHY } } }
I20260809 23:15:23.547336 20880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:15:23.545617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60006 (local address 127.2.76.195:33883)
0809 23:15:23.545645 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0809 23:15:23.545647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:15:23.545695 (+ 48us) server_negotiation.cc:407] Connection header received
0809 23:15:23.545756 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:15:23.545756 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:15:23.545768 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:15:23.545810 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:15:23.545893 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.546244 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.546819 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:15:23.546960 (+ 141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:15:23.547003 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:15:23.547146 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:15:23.547214 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:15:23.547261 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 23:15:23.547272 (+ 11us) server_negotiation.cc:299] Negotiation successful
0809 23:15:23.547284 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 23:15:23.548712 20576 raft_consensus.cc:1261] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71 [term 1 FOLLOWER]: Refusing update from remote peer 4091e34c8fbe49ddab3c47d6a759b2fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:15:23.548775 20365 raft_consensus.cc:1261] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca [term 1 FOLLOWER]: Refusing update from remote peer 4091e34c8fbe49ddab3c47d6a759b2fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:15:23.548977 20896 consensus_queue.cc:1035] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c00e546255f4456a481e1f224b52b71" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 38023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:15:23.549062 20888 consensus_queue.cc:1035] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "360a5512821f4a20917b07fac6ddbeca" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:15:23.550994 20902 mvcc.cc:205] Tried to move back new op lower bound from 7316755757253627904 to 7316755757220605952. Current Snapshot: MvccSnapshot[applied={T|T < 7316755757253627904}]
I20260809 23:15:23.551180 20906 mvcc.cc:205] Tried to move back new op lower bound from 7316755757253627904 to 7316755757220605952. Current Snapshot: MvccSnapshot[applied={T|T < 7316755757253627904}]
I20260809 23:15:23.551173 20905 mvcc.cc:205] Tried to move back new op lower bound from 7316755757253627904 to 7316755757220605952. Current Snapshot: MvccSnapshot[applied={T|T < 7316755757253627904}]
I20260809 23:15:23.553498 20127 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:15:23.553591 20127 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:15:23.554237 20127 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f1174dcd23b41629187a557c834fe04: Sending DeleteTablet for 3 replicas of tablet 3871387002fc436f813a1cb0235b5e0d
I20260809 23:15:23.554512 20543 tablet_service.cc:1514] Processing DeleteTablet for tablet 3871387002fc436f813a1cb0235b5e0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:15:23 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:60548
I20260809 23:15:23.554540 20750 tablet_service.cc:1514] Processing DeleteTablet for tablet 3871387002fc436f813a1cb0235b5e0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:15:23 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:59992
I20260809 23:15:23.554589 20332 tablet_service.cc:1514] Processing DeleteTablet for tablet 3871387002fc436f813a1cb0235b5e0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:15:23 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:37130
I20260809 23:15:23.554802 20921 tablet_replica.cc:330] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca: stopping tablet replica
I20260809 23:15:23.554930 20921 raft_consensus.cc:2229] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:15:23.554965 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 20224
I20260809 23:15:23.555212 20921 raft_consensus.cc:2258] T 3871387002fc436f813a1cb0235b5e0d P 360a5512821f4a20917b07fac6ddbeca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:15:23.576417 20920 tablet_replica.cc:330] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71: stopping tablet replica
I20260809 23:15:23.576599 20920 raft_consensus.cc:2229] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:15:23.576748 20920 raft_consensus.cc:2258] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:15:23.577248 20919 tablet_replica.cc:330] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd: stopping tablet replica
I20260809 23:15:23.577375 20919 raft_consensus.cc:2229] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:15:23.577596 20919 raft_consensus.cc:2258] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:15:23.578907 20919 ts_tablet_manager.cc:1905] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:15:23.582288 20920 ts_tablet_manager.cc:1905] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:15:23.583694 20920 ts_tablet_manager.cc:1918] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:15:23.583752 20920 log.cc:1196] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/3871387002fc436f813a1cb0235b5e0d
I20260809 23:15:23.583974 20920 ts_tablet_manager.cc:1939] T 3871387002fc436f813a1cb0235b5e0d P 4c00e546255f4456a481e1f224b52b71: Deleting consensus metadata
I20260809 23:15:23.584271 20919 ts_tablet_manager.cc:1918] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:15:23.584434 20919 log.cc:1196] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/3871387002fc436f813a1cb0235b5e0d
I20260809 23:15:23.584420 20102 catalog_manager.cc:4841] TS 4c00e546255f4456a481e1f224b52b71 (127.2.76.194:38023): tablet 3871387002fc436f813a1cb0235b5e0d (table test-table [id=14642944bcc14a3f91e0eb0bae142d71]) successfully deleted
I20260809 23:15:23.584690 20919 ts_tablet_manager.cc:1939] T 3871387002fc436f813a1cb0235b5e0d P 4091e34c8fbe49ddab3c47d6a759b2fd: Deleting consensus metadata
I20260809 23:15:23.585039 20104 catalog_manager.cc:4841] TS 4091e34c8fbe49ddab3c47d6a759b2fd (127.2.76.195:33883): tablet 3871387002fc436f813a1cb0235b5e0d (table test-table [id=14642944bcc14a3f91e0eb0bae142d71]) successfully deleted
W20260809 23:15:23.587018 20103 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:33983 (error 108)
W20260809 23:15:23.587906 20872 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:15:23.587709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:33983 (local address 127.0.0.1:37146)
0809 23:15:23.587765 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 23:15:23.587776 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:15:23.587823 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:15:23.587870 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:33983: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 23:15:23.588049 20103 catalog_manager.cc:4584] TS 360a5512821f4a20917b07fac6ddbeca (127.2.76.193:33983): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3871387002fc436f813a1cb0235b5e0d: Network error: Client connection negotiation failed: client connection to 127.2.76.193:33983: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260809 23:15:23.588411 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 20441
I20260809 23:15:23.604658 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 20658
I20260809 23:15:23.611577 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 20075
2026-08-09T23:15:23Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1956 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:15:23 dist-test-slave-7vjv krb5kdc[20925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:15:23 dist-test-slave-7vjv krb5kdc[20925](info): set up 2 sockets
Aug 09 23:15:23 dist-test-slave-7vjv krb5kdc[20925](info): commencing operation
krb5kdc: starting...
W20260809 23:15:24.762333 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:15:24 dist-test-slave-7vjv krb5kdc[20925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:15:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:15:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.76.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.76.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/rangeradmin_127.2.76.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/rangeradmin_127.2.76.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.76.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.76.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/rangerlookup_127.2.76.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/rangerlookup_127.2.76.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.76.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.76.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/HTTP_127.2.76.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/HTTP_127.2.76.212@KRBTEST.COM.keytab.
I20260809 23:15:24.902182 2355 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskvJxnr9/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/postgres -l logfile start
2026-08-09 23:15:27.092 UTC [20965] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 23:15:27.092 UTC [20965] LOG: listening on IPv4 address "127.2.76.212", port 40761
2026-08-09 23:15:27.095 UTC [20965] LOG: listening on Unix socket "/tmp/.s.PGSQL.40761"
2026-08-09 23:15:27.130 UTC [20975] LOG: database system was shut down at 2026-08-09 23:15:25 UTC
2026-08-09 23:15:27.134 UTC [20965] LOG: database system is ready to accept connections
I20260809 23:15:28.177654 2355 mini_postgres.cc:96] Postgres bound to 40761
2026-08-09 23:15:28.185 UTC [20988] FATAL: database "slave" does not exist
127.2.76.212:40761 - accepting connections
I20260809 23:15:28.186400 2355 mini_ranger.cc:161] Starting Ranger...
I20260809 23:15:28.200843 2355 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 23:15:28.606871 2355 mini_ranger.cc:87] Created ranger Postgres database
I20260809 23:15:28.606938 2355 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-admin
2026-08-09 23:15:28,884 [I] DB FLAVOR :POSTGRES
2026-08-09 23:15:28,885 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 23:15:28,885 [I] Checking connection..
2026-08-09 23:15:28,885 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 23:15:29,124 [I] Checking connection passed.
2026-08-09 23:15:29,125 [I] --------- Verifying version history table ---------
2026-08-09 23:15:29,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:15:29,361 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 23:15:29,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:15:29,592 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 23:15:29,592 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 23:15:29,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 23:15:29.822 UTC [21099] WARNING: there is no transaction in progress
2026-08-09 23:15:29,831 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 23:15:29,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:15:30,067 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 23:15:30,068 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 23:15:30,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 23:15:30,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 23:15:30,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7vjv.c.gcp-upstream.internal','N') ;"
2026-08-09 23:15:30,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 23:15:31,019 [I] Table x_portal_user does not exist in database ranger
2026-08-09 23:15:31,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 23:15:31,255 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 23:15:31,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 23:15:31,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 23:15:31,703 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 23:15:31,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 23:15:32.534 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:32.549 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:32.564 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:32.578 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:32.596 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:32.609 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:32.655 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:32.661 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:32.672 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:32.681 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:32.690 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:32.701 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:32.709 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:32.715 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:32.720 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:32.726 UTC [21288] WARNING: there is no transaction in progress
2026-08-09 23:15:33,275 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 23:15:33,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 23:15:33,508 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 23:15:33,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 23:15:33,717 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 23:15:33,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 23:15:33,927 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 23:15:33,928 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 23:15:34,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 23:15:34,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7vjv.c.gcp-upstream.internal';"
2026-08-09 23:15:34,603 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 23:15:34,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 23:15:34,812 [I] DB_PATCHES have already been applied
I20260809 23:15:34.817749 2355 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-admin:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskvJxnr9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 23:15:34.817806 2355 mini_ranger.cc:193] Using host: 127.2.76.212
I20260809 23:15:34.821286 2355 mini_ranger.cc:227] Ranger admin URL: http://127.2.76.212:40599
Aug 09, 2026 11:15:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 11:15:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 11:15:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 11:15:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 11:15:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 11:15:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 11:15:35 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 23:15:36.074743 2355 mini_ranger.cc:160] Time spent starting Ranger: real 7.888s user 0.001s sys 0.009s
Aug 09, 2026 11:15:54 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 11:15:54 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 09, 2026 11:15:54 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 11:15:54 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 11:15:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 11:15:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 11:15:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 11:15:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 11:15:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 11:15:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 11:15:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 11:15:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 11:15:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 11:15:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 11:15:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 11:15:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 11:15:54 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 11:15:55 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 23:15:55.647132 2355 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.76.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.76.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/rangerkms_127.2.76.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/rangerkms_127.2.76.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.76.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.76.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.76.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/HTTP_127.2.76.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.76.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/HTTP_127.2.76.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 23:15:55 dist-test-slave-7vjv krb5kdc[20925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317355, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 23:15:55.717150 2355 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 23:15:55.725754 2355 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 23:15:56.471565 2355 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 23:15:56.471650 2355 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-kms
I20260809 23:15:56.471668 2355 mini_ranger_kms.cc:227] Using postgres at 127.2.76.212:40761
2026-08-09 23:15:56,690 [I] DB FLAVOR :POSTGRES
2026-08-09 23:15:56,690 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 23:15:56,690 [I] Checking connection
2026-08-09 23:15:56,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 23:15:56,923 [I] connection success
2026-08-09 23:15:56,924 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 23:15:56,924 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 23:15:56,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 23:15:57,160 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 23:15:57,160 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 23:15:57,160 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 23:15:57,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:40761/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 23:15:57,387 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 23:15:57.836277 2355 mini_ranger_kms.cc:310] Created kms service
I20260809 23:15:58.016140 2355 mini_ranger_kms.cc:326] Created kudu user
I20260809 23:15:58.056231 2355 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 23:15:58 dist-test-slave-7vjv krb5kdc[20925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317355, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:15:58.397925 2355 mini_ranger_kms.cc:384] Added ranger policy
I20260809 23:15:58.398200 2355 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-kms:/tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 23:15:58.398255 2355 mini_ranger_kms.cc:240] Using host: 127.2.76.212
I20260809 23:15:58.402815 2355 mini_ranger_kms.cc:276] Ranger KMS PID: 21581
I20260809 23:15:58.402909 2355 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.76.212:41693
Aug 09, 2026 11:15:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 11:15:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 11:15:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 11:15:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 11:15:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 11:15:59 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 23:16:00 dist-test-slave-7vjv krb5kdc[20925](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786317360, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.76.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:00.917505 2355 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.200s user 0.000s sys 0.204s
I20260809 23:16:00.917604 2355 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 23:16:00.917671 2355 mini_ranger_kms.cc:401] 127.2.76.212:41693/kms/v1/keys
Aug 09 23:16:01 dist-test-slave-7vjv krb5kdc[20925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317360, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.76.212@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
Aug 09 23:16:01 dist-test-slave-7vjv krb5kdc[20925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317360, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.76.212@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
Aug 09 23:16:03 dist-test-slave-7vjv krb5kdc[20925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:16:03.466265 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:40105
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:44949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:40105
--ranger_config_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.76.212:41693/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:03.541491 21667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:03.541661 21667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:03.541682 21667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:03.542661 21667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:16:03.542704 21667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:03.542718 21667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:03.542730 21667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:16:03.542742 21667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:16:03.544215 21667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:44949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.76.212:41693/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:40105
--ranger_config_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:40105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:03.544448 21667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:03.544687 21667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:03.549134 21686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:03.549146 21679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:03.549281 21667 server_base.cc:1034] running on GCE node
W20260809 23:16:03.549146 21682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:03.549680 21667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:03.549969 21667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:03.551241 21667 hybrid_clock.cc:648] HybridClock initialized: now 1786317363551218 us; error 38 us; skew 500 ppm
Aug 09 23:16:03 dist-test-slave-7vjv krb5kdc[20925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317363, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:03.556654 21667 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:16:03.557273 21667 webserver.cc:460] Webserver started at http://127.2.76.254:42753/ using document root <none> and password file <none>
I20260809 23:16:03.557519 21667 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:03.557574 21667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:03.557710 21667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:16:03 dist-test-slave-7vjv krb5kdc[20925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317363, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:03.600153 21667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "2368643443a54b32a046f00daedae538"
format_stamp: "Formatted at 2026-08-09 23:16:03 on dist-test-slave-7vjv"
server_key: "22964d0c05ce23a3cdae0e6c45ce05d6"
server_key_iv: "8a60678b2abd888663e264673e4d93c3"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:03.600660 21667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "2368643443a54b32a046f00daedae538"
format_stamp: "Formatted at 2026-08-09 23:16:03 on dist-test-slave-7vjv"
server_key: "22964d0c05ce23a3cdae0e6c45ce05d6"
server_key_iv: "8a60678b2abd888663e264673e4d93c3"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:03.602111 21667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:16:03.615379 21699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:03.615604 21667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:16:03.615717 21667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "2368643443a54b32a046f00daedae538"
format_stamp: "Formatted at 2026-08-09 23:16:03 on dist-test-slave-7vjv"
server_key: "22964d0c05ce23a3cdae0e6c45ce05d6"
server_key_iv: "8a60678b2abd888663e264673e4d93c3"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:03.615808 21667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:03.640851 21667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:03.642668 21667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:03.642822 21667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:03.647891 21667 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:40105
I20260809 23:16:03.647923 21787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:40105 every 8 connection(s)
I20260809 23:16:03.648322 21667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:16:03.649075 21788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:03.651207 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 21667
I20260809 23:16:03.651302 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:16:03.651468 2355 external_mini_cluster.cc:1428] Setting key 08bc67262fe40989e78424466fe42ffc
I20260809 23:16:03.652344 21788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538: Bootstrap starting.
I20260809 23:16:03.653138 21788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:03.653483 21788 log.cc:824] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:03.654328 21788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538: No bootstrap required, opened a new log
I20260809 23:16:03.656880 21788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2368643443a54b32a046f00daedae538" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40105 } }
I20260809 23:16:03.657040 21788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:03.657073 21788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2368643443a54b32a046f00daedae538, State: Initialized, Role: FOLLOWER
I20260809 23:16:03.657236 21788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2368643443a54b32a046f00daedae538" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40105 } }
I20260809 23:16:03.657330 21788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:16:03.657392 21788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:16:03.657444 21788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 23:16:03 dist-test-slave-7vjv krb5kdc[20925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:03.658062 21788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2368643443a54b32a046f00daedae538" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40105 } }
I20260809 23:16:03.658190 21788 leader_election.cc:302] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2368643443a54b32a046f00daedae538; no voters:
I20260809 23:16:03.658370 21788 leader_election.cc:288] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:16:03.658629 21788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:16:03.660379 21788 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 23:16:03.660876 21802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:16:03.661243 21796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:03.651946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35018 (local address 127.2.76.254:40105)
0809 23:16:03.652153 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:03.652162 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:03.652195 (+ 33us) server_negotiation.cc:407] Connection header received
0809 23:16:03.652663 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:03.652669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:03.652716 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:03.652854 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:03.653461 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:03.654488 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:03.655147 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:03.655526 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:03.655633 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:03.657980 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:03.657984 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:03.657984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:03.658005 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:03.659703 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:03.660073 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:03.660075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:03.660076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:03.660180 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:03.660378 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:03.660378 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:03.660379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:03.660674 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:03.660817 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:03.660911 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 23:16:03.661020 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":91,"threads_started":1}
I20260809 23:16:03.661068 21802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [term 1 LEADER]: Becoming Leader. State: Replica: 2368643443a54b32a046f00daedae538, State: Running, Role: LEADER
I20260809 23:16:03.661610 21802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2368643443a54b32a046f00daedae538" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40105 } }
I20260809 23:16:03.661923 21802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2368643443a54b32a046f00daedae538. Latest consensus state: current_term: 1 leader_uuid: "2368643443a54b32a046f00daedae538" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2368643443a54b32a046f00daedae538" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40105 } } }
I20260809 23:16:03.662001 21802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:03.665521 21832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:16:03.665505 21826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2368643443a54b32a046f00daedae538" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2368643443a54b32a046f00daedae538" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 40105 } } }
I20260809 23:16:03.665755 21826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538 [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:03.666465 21832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:16:03.668035 21832 catalog_manager.cc:1347] Generated new cluster ID: b43f93ce480948b591a27ececd0038f8
I20260809 23:16:03.668076 21832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:16:03.675300 21832 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:16:03.675992 21832 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:16:03.682595 21832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538: Generated new TSK 0
I20260809 23:16:03.682768 21832 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:16:04.372416 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.76.212:41693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40105
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:44949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:04.466976 21873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:04.467188 21873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:04.467228 21873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:04.468432 21873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:04.468490 21873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:04.468538 21873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:16:04.469832 21873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:44949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.76.212:41693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:04.470074 21873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:04.470289 21873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:04.475306 21888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:04.475327 21885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:04.475438 21873 server_base.cc:1034] running on GCE node
W20260809 23:16:04.475337 21886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:04.475766 21873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:04.476023 21873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:04.477209 21873 hybrid_clock.cc:648] HybridClock initialized: now 1786317364477208 us; error 46 us; skew 500 ppm
Aug 09 23:16:04 dist-test-slave-7vjv krb5kdc[20925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317364, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:04.479796 21873 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:16:04.480294 21873 webserver.cc:460] Webserver started at http://127.2.76.193:40769/ using document root <none> and password file <none>
I20260809 23:16:04.480487 21873 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:04.480552 21873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:04.480667 21873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:16:04 dist-test-slave-7vjv krb5kdc[20925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317364, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:04.510921 21873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "161b097116dc4d329fe8c7e52bc52ea3"
format_stamp: "Formatted at 2026-08-09 23:16:04 on dist-test-slave-7vjv"
server_key: "691799e51b4c6146a3875cee536e61d8"
server_key_iv: "734c8e3c59ab3b5458ee70efc2854a20"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:04.511463 21873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "161b097116dc4d329fe8c7e52bc52ea3"
format_stamp: "Formatted at 2026-08-09 23:16:04 on dist-test-slave-7vjv"
server_key: "691799e51b4c6146a3875cee536e61d8"
server_key_iv: "734c8e3c59ab3b5458ee70efc2854a20"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:04.513746 21873 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 23:16:04.527768 21908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:04.527973 21873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 23:16:04.528070 21873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "161b097116dc4d329fe8c7e52bc52ea3"
format_stamp: "Formatted at 2026-08-09 23:16:04 on dist-test-slave-7vjv"
server_key: "691799e51b4c6146a3875cee536e61d8"
server_key_iv: "734c8e3c59ab3b5458ee70efc2854a20"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:04.528159 21873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:04.544538 21873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:04.546639 21873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:04.546820 21873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:04.547080 21873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:04.547462 21873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:04.547550 21873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:04.547639 21873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:04.547706 21873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:04.554070 21873 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:45501
I20260809 23:16:04.554488 21873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:16:04.557178 22093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:45501 every 8 connection(s)
I20260809 23:16:04.559755 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 21873
I20260809 23:16:04.559852 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:16:04.560061 2355 external_mini_cluster.cc:1428] Setting key 433db3cf31664b6c89ad76c479444bf2
Aug 09 23:16:04 dist-test-slave-7vjv krb5kdc[20925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317364, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:04.567591 22098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:04.556729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40105 (local address 127.2.76.193:39689)
0809 23:16:04.557424 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:04.557463 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:04.558033 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:04.558522 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:04.558526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:04.558742 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:04.558991 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:04.558997 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:04.560048 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:04.560049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:04.561235 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:04.561238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:04.561752 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:04.561753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:04.561885 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:04.562439 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:04.562449 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:04.564171 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:04.566104 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:04.566110 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:04.566112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:04.566404 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:04.566596 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:04.566597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:04.566597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:04.566722 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:04.566985 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:04.566991 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:04.567129 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 23:16:04.567228 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 23:16:04.567374 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":95,"threads_started":1}
I20260809 23:16:04.568210 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:04.556763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:39689 (local address 127.2.76.254:40105)
0809 23:16:04.558302 (+ 1539us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:04.558304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:04.558312 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:16:04.558349 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:04.558350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:04.558363 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:04.559080 (+ 717us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:04.559120 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:04.559953 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:04.561343 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:04.561632 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:04.561684 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:04.564271 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:04.564273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:04.564273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:04.564280 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:04.566019 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:04.566480 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:04.566481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:04.566481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:04.566540 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:04.566784 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:04.566785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:04.566785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:04.566903 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:04.566960 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:04.568028 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0809 23:16:04.568119 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1504,"thread_start_us":88,"threads_started":1}
I20260809 23:16:04.568684 22095 heartbeater.cc:347] Connected to a master server at 127.2.76.254:40105
I20260809 23:16:04.568759 22095 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:04.568910 22095 heartbeater.cc:511] Master 127.2.76.254:40105 requested a full tablet report, sending...
I20260809 23:16:04.569425 21730 ts_manager.cc:194] Registered new tserver with Master: 161b097116dc4d329fe8c7e52bc52ea3 (127.2.76.193:45501)
I20260809 23:16:04.570171 21730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:39689
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:16:04.596071 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.76.212:41693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40105
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:44949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:16:04 dist-test-slave-7vjv krb5kdc[20925](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786317364, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 23:16:04.699136 22108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:04.699334 22108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:04.699370 22108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:04.700515 22108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:04.700575 22108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:04.700616 22108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:16:04.701998 22108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:44949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.76.212:41693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:04.702217 22108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:04.702440 22108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:04.707438 22117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:04.707588 22108 server_base.cc:1034] running on GCE node
W20260809 23:16:04.707685 22119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:04.707515 22116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:04.707957 22108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:04.708220 22108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:04.709371 22108 hybrid_clock.cc:648] HybridClock initialized: now 1786317364709355 us; error 30 us; skew 500 ppm
Aug 09 23:16:04 dist-test-slave-7vjv krb5kdc[20925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317364, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:04.711692 22108 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:16:04.712119 22108 webserver.cc:460] Webserver started at http://127.2.76.194:39151/ using document root <none> and password file <none>
I20260809 23:16:04.712339 22108 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:04.712380 22108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:04.712502 22108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:16:04 dist-test-slave-7vjv krb5kdc[20925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317364, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:04.726818 22108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "167a2393756f41ff9b5adc62ee35e9cb"
format_stamp: "Formatted at 2026-08-09 23:16:04 on dist-test-slave-7vjv"
server_key: "94032d4f98374746ee19d1d01c9443e3"
server_key_iv: "456ff4414b46ba92a9597af2ef28728c"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:04.727284 22108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "167a2393756f41ff9b5adc62ee35e9cb"
format_stamp: "Formatted at 2026-08-09 23:16:04 on dist-test-slave-7vjv"
server_key: "94032d4f98374746ee19d1d01c9443e3"
server_key_iv: "456ff4414b46ba92a9597af2ef28728c"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:04.732511 22108 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.003s
I20260809 23:16:04.748493 22137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:04.748888 22108 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260809 23:16:04.748997 22108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "167a2393756f41ff9b5adc62ee35e9cb"
format_stamp: "Formatted at 2026-08-09 23:16:04 on dist-test-slave-7vjv"
server_key: "94032d4f98374746ee19d1d01c9443e3"
server_key_iv: "456ff4414b46ba92a9597af2ef28728c"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:04.749068 22108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:04.760398 22108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:04.762524 22108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:04.762707 22108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:04.762949 22108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:04.763415 22108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:04.763499 22108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:04.763571 22108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:04.763624 22108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:04.771029 22108 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:45773
I20260809 23:16:04.771510 22108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:16:04.778527 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 22108
I20260809 23:16:04.778635 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:16:04.778836 2355 external_mini_cluster.cc:1428] Setting key be290765b21d6d6cc433fbfa36be69c9
Aug 09 23:16:04 dist-test-slave-7vjv krb5kdc[20925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317364, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:04.783249 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:04.773374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:48943 (local address 127.2.76.254:40105)
0809 23:16:04.773420 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:04.773421 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:04.773728 (+ 307us) server_negotiation.cc:407] Connection header received
0809 23:16:04.774476 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:04.774477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:04.774492 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:04.774524 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:04.775331 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:04.776046 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:04.777419 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:04.777672 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:04.777713 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:04.780211 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:04.780212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:04.780213 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:04.780220 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:04.781811 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:04.782281 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:04.782282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:04.782282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:04.782351 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:04.782602 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:04.782603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:04.782603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:04.782718 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:04.782760 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:04.783089 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 23:16:04.783171 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:16:04.772568 22331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:45773 every 8 connection(s)
I20260809 23:16:04.783883 22334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:04.773093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40105 (local address 127.2.76.194:48943)
0809 23:16:04.773606 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:04.773646 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:04.774328 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:04.774704 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:04.774708 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:04.774946 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:04.775203 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:04.775209 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:04.776208 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:04.776209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:04.777287 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:04.777291 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:04.777795 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:04.777796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:04.777920 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:04.778417 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:04.778426 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:04.780082 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:04.781916 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:04.781922 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:04.781924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:04.782195 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:04.782419 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:04.782420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:04.782420 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:04.782535 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:04.782806 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:04.782812 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:04.782948 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 23:16:04.783551 (+ 603us) client_negotiation.cc:240] Negotiation successful
0809 23:16:04.783689 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":85,"threads_started":1}
I20260809 23:16:04.784802 22332 heartbeater.cc:347] Connected to a master server at 127.2.76.254:40105
I20260809 23:16:04.784884 22332 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:04.785054 22332 heartbeater.cc:511] Master 127.2.76.254:40105 requested a full tablet report, sending...
I20260809 23:16:04.785437 21730 ts_manager.cc:194] Registered new tserver with Master: 167a2393756f41ff9b5adc62ee35e9cb (127.2.76.194:45773)
I20260809 23:16:04.785856 21730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:48943
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:16:04.812393 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.76.212:41693/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40105
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:44949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:04.924666 22339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:04.924995 22339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:04.925087 22339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:04.926530 22339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:04.926631 22339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:04.926678 22339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:16:04.928735 22339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:44949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.76.212:41693/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:40105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:04.929188 22339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:04.929479 22339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:04.946094 22347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:04.946098 22345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:04.946498 22344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:04.946607 22339 server_base.cc:1034] running on GCE node
I20260809 23:16:04.946831 22339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:04.947073 22339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:04.948211 22339 hybrid_clock.cc:648] HybridClock initialized: now 1786317364948195 us; error 32 us; skew 500 ppm
Aug 09 23:16:04 dist-test-slave-7vjv krb5kdc[20925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317364, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:04.951201 22339 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:16:04.951701 22339 webserver.cc:460] Webserver started at http://127.2.76.195:39311/ using document root <none> and password file <none>
I20260809 23:16:04.951944 22339 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:04.952009 22339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:04.952261 22339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:16:04 dist-test-slave-7vjv krb5kdc[20925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317364, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:04.965353 22339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "4e5335c78c964517be89c2ce0b3df4ba"
format_stamp: "Formatted at 2026-08-09 23:16:04 on dist-test-slave-7vjv"
server_key: "772e009a02245373887881916d52c6ea"
server_key_iv: "1f7e28c4416b2405fd33f9c9adcac91c"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:04.966064 22339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "4e5335c78c964517be89c2ce0b3df4ba"
format_stamp: "Formatted at 2026-08-09 23:16:04 on dist-test-slave-7vjv"
server_key: "772e009a02245373887881916d52c6ea"
server_key_iv: "1f7e28c4416b2405fd33f9c9adcac91c"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:04.967911 22339 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 23:16:04.990540 22359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:04.990741 22339 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:16:04.990829 22339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "4e5335c78c964517be89c2ce0b3df4ba"
format_stamp: "Formatted at 2026-08-09 23:16:04 on dist-test-slave-7vjv"
server_key: "772e009a02245373887881916d52c6ea"
server_key_iv: "1f7e28c4416b2405fd33f9c9adcac91c"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:04.990926 22339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:05.002789 22339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:05.005327 22339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:05.005524 22339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:05.005825 22339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:05.006249 22339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:05.006336 22339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:05.006410 22339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:05.006470 22339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:05.015791 22339 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:41845
I20260809 23:16:05.016290 22339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:16:05.018987 22562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:41845 every 8 connection(s)
I20260809 23:16:05.025892 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 22339
I20260809 23:16:05.025978 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:16:05.026139 2355 external_mini_cluster.cc:1428] Setting key 5d042ab0280e7959a252abbb4778ecc0
Aug 09 23:16:05 dist-test-slave-7vjv krb5kdc[20925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317364, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:05.031448 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.019927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:42879 (local address 127.2.76.254:40105)
0809 23:16:05.019968 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:05.019970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:05.021067 (+ 1097us) server_negotiation.cc:407] Connection header received
0809 23:16:05.021686 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:05.021688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:05.021699 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:05.021726 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:05.022491 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.023172 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.024607 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.024884 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.024924 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:05.028573 (+ 3649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:05.028574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:05.028574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:05.028581 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:05.030086 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:05.030522 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:05.030523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:05.030524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:05.030577 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:05.030806 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:05.030807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:05.030807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:05.030921 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:05.030966 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:05.031263 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 23:16:05.031342 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:16:05.032076 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.019657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:40105 (local address 127.2.76.195:42879)
0809 23:16:05.020944 (+ 1287us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:05.020987 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:05.021570 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:05.021889 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:05.021893 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:05.022111 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:05.022346 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:05.022351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.023277 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.023278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:05.024513 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:05.024516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.025006 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.025006 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:05.025133 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:05.025679 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:05.025688 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:05.028474 (+ 2786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:05.030172 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:05.030178 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:05.030180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:05.030443 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:05.030634 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:05.030634 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:05.030635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:05.030739 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:05.030995 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:05.031000 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:05.031134 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 23:16:05.031737 (+ 603us) client_negotiation.cc:240] Negotiation successful
0809 23:16:05.031884 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1246,"thread_start_us":74,"threads_started":1}
I20260809 23:16:05.032641 22563 heartbeater.cc:347] Connected to a master server at 127.2.76.254:40105
I20260809 23:16:05.032720 22563 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:05.032876 22563 heartbeater.cc:511] Master 127.2.76.254:40105 requested a full tablet report, sending...
I20260809 23:16:05.033242 21730 ts_manager.cc:194] Registered new tserver with Master: 4e5335c78c964517be89c2ce0b3df4ba (127.2.76.195:41845)
I20260809 23:16:05.033646 21730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:42879
I20260809 23:16:05.037534 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:16:05.051069 22099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.038786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35026 (local address 127.2.76.254:40105)
0809 23:16:05.039126 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:05.039128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:05.040273 (+ 1145us) server_negotiation.cc:407] Connection header received
0809 23:16:05.040367 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:05.040367 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:05.040377 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:05.040404 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:05.041152 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.041843 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.043411 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.043665 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.043701 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:05.047483 (+ 3782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:05.047484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:05.047485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:05.047491 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:05.049116 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:05.050339 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:05.050340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:05.050341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:05.050393 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:05.050565 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:05.050566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:05.050566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:05.050669 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:05.050733 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:05.050924 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 23:16:05.050992 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318}
I20260809 23:16:05.053097 21730 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:16:05.055130 21730 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:16:05.080292 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.070431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:45501 (local address 127.0.0.1:38096)
0809 23:16:05.072985 (+ 2554us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:05.072993 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:05.073037 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:05.073368 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:05.073370 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:05.073371 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:05.073430 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:05.073431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.074646 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.074648 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:05.075932 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:05.075935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.080042 (+ 4107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.080042 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:05.080103 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:05.080131 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 23:16:05.080183 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 23:16:05.080224 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2540,"thread_start_us":196,"threads_started":1}
I20260809 23:16:05.081511 22588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.071121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:41845 (local address 127.0.0.1:36784)
0809 23:16:05.073259 (+ 2138us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:05.073267 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:05.073299 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:05.076307 (+ 3008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:05.076307 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:05.076332 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:05.076367 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:05.076368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.077553 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.077554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:05.078650 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:05.078653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.081326 (+ 2673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.081327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:05.081376 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:05.081379 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:16:05.081421 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 23:16:05.081454 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2125,"thread_start_us":172,"threads_started":1}
I20260809 23:16:05.082964 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.070792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38096 (local address 127.2.76.193:45501)
0809 23:16:05.072939 (+ 2147us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:05.072943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:05.073079 (+ 136us) server_negotiation.cc:407] Connection header received
0809 23:16:05.073112 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:05.073114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:05.073129 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:05.073186 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:05.073509 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.074550 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.078745 (+ 4195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.079463 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.079538 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:05.079605 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:05.082854 (+ 3249us) server_negotiation.cc:299] Negotiation successful
0809 23:16:05.082896 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2113,"thread_start_us":110,"threads_started":1}
I20260809 23:16:05.083808 21978 tablet_service.cc:1467] Processing CreateTablet for tablet 50c242fd679643968ad7e6bb71732909 (DEFAULT_TABLE table=test-table [id=066782826f6347fda3f4b63f36ed7c6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:16:05.084213 21978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50c242fd679643968ad7e6bb71732909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:05.088163 22589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.071469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36784 (local address 127.2.76.195:41845)
0809 23:16:05.076084 (+ 4615us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:05.076088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:05.076098 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:16:05.076151 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:05.076153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:05.076169 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:05.076233 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:05.076434 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.077453 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.080595 (+ 3142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.081166 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.081225 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:05.081285 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:05.088022 (+ 6737us) server_negotiation.cc:299] Negotiation successful
0809 23:16:05.088069 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4579,"thread_start_us":153,"threads_started":1}
I20260809 23:16:05.090912 22435 tablet_service.cc:1467] Processing CreateTablet for tablet 50c242fd679643968ad7e6bb71732909 (DEFAULT_TABLE table=test-table [id=066782826f6347fda3f4b63f36ed7c6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:16:05.091228 22435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50c242fd679643968ad7e6bb71732909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:05.094847 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.070078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:45773 (local address 127.0.0.1:36924)
0809 23:16:05.072660 (+ 2582us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:05.072699 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:05.072769 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:05.079931 (+ 7162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:05.079932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:05.079934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:05.079970 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:05.079971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.088416 (+ 8445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.088417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:05.089488 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:05.089491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.094615 (+ 5124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.094616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:05.094681 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:05.094697 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 23:16:05.094724 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 23:16:05.094779 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2549,"thread_start_us":202,"threads_started":1}
I20260809 23:16:05.095388 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.070246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36924 (local address 127.2.76.194:45773)
0809 23:16:05.079702 (+ 9456us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:05.079706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:05.079715 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:16:05.079748 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:05.079750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:05.079763 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:05.079826 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:05.081574 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.082387 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.089623 (+ 7236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.090290 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.095140 (+ 4850us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:05.095233 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:05.095259 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 23:16:05.095299 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9415,"thread_start_us":102,"threads_started":1}
I20260809 23:16:05.096127 22212 tablet_service.cc:1467] Processing CreateTablet for tablet 50c242fd679643968ad7e6bb71732909 (DEFAULT_TABLE table=test-table [id=066782826f6347fda3f4b63f36ed7c6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:16:05.096410 22212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50c242fd679643968ad7e6bb71732909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:05.099738 22590 tablet_bootstrap.cc:492] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3: Bootstrap starting.
I20260809 23:16:05.100663 22590 tablet_bootstrap.cc:654] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:05.100864 22594 tablet_bootstrap.cc:492] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba: Bootstrap starting.
I20260809 23:16:05.101783 22594 tablet_bootstrap.cc:654] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:05.102259 22590 log.cc:824] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:05.102602 22594 log.cc:824] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:05.103063 22597 tablet_bootstrap.cc:492] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb: Bootstrap starting.
I20260809 23:16:05.103946 22597 tablet_bootstrap.cc:654] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:05.104349 22597 log.cc:824] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:05.105355 22590 tablet_bootstrap.cc:492] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3: No bootstrap required, opened a new log
I20260809 23:16:05.105470 22590 ts_tablet_manager.cc:1397] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260809 23:16:05.108062 22590 raft_consensus.cc:348] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167a2393756f41ff9b5adc62ee35e9cb" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 45773 } } peers { permanent_uuid: "4e5335c78c964517be89c2ce0b3df4ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 41845 } } peers { permanent_uuid: "161b097116dc4d329fe8c7e52bc52ea3" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 45501 } }
I20260809 23:16:05.108304 22590 raft_consensus.cc:374] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:05.108408 22590 raft_consensus.cc:729] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 161b097116dc4d329fe8c7e52bc52ea3, State: Initialized, Role: FOLLOWER
I20260809 23:16:05.108587 22590 consensus_queue.cc:260] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167a2393756f41ff9b5adc62ee35e9cb" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 45773 } } peers { permanent_uuid: "4e5335c78c964517be89c2ce0b3df4ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 41845 } } peers { permanent_uuid: "161b097116dc4d329fe8c7e52bc52ea3" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 45501 } }
I20260809 23:16:05.108889 22590 ts_tablet_manager.cc:1428] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 23:16:05.109032 22095 heartbeater.cc:503] Master 127.2.76.254:40105 was elected leader, sending a full tablet report...
I20260809 23:16:05.110092 22594 tablet_bootstrap.cc:492] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba: No bootstrap required, opened a new log
I20260809 23:16:05.110193 22594 ts_tablet_manager.cc:1397] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260809 23:16:05.111865 22594 raft_consensus.cc:348] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167a2393756f41ff9b5adc62ee35e9cb" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 45773 } } peers { permanent_uuid: "4e5335c78c964517be89c2ce0b3df4ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 41845 } } peers { permanent_uuid: "161b097116dc4d329fe8c7e52bc52ea3" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 45501 } }
I20260809 23:16:05.112046 22594 raft_consensus.cc:374] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:05.112139 22594 raft_consensus.cc:729] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e5335c78c964517be89c2ce0b3df4ba, State: Initialized, Role: FOLLOWER
I20260809 23:16:05.112295 22594 consensus_queue.cc:260] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167a2393756f41ff9b5adc62ee35e9cb" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 45773 } } peers { permanent_uuid: "4e5335c78c964517be89c2ce0b3df4ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 41845 } } peers { permanent_uuid: "161b097116dc4d329fe8c7e52bc52ea3" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 45501 } }
I20260809 23:16:05.112563 22594 ts_tablet_manager.cc:1428] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:16:05.112692 22563 heartbeater.cc:503] Master 127.2.76.254:40105 was elected leader, sending a full tablet report...
I20260809 23:16:05.113912 22597 tablet_bootstrap.cc:492] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb: No bootstrap required, opened a new log
I20260809 23:16:05.114012 22597 ts_tablet_manager.cc:1397] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20260809 23:16:05.115809 22597 raft_consensus.cc:348] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167a2393756f41ff9b5adc62ee35e9cb" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 45773 } } peers { permanent_uuid: "4e5335c78c964517be89c2ce0b3df4ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 41845 } } peers { permanent_uuid: "161b097116dc4d329fe8c7e52bc52ea3" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 45501 } }
I20260809 23:16:05.116029 22597 raft_consensus.cc:374] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:05.116107 22597 raft_consensus.cc:729] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 167a2393756f41ff9b5adc62ee35e9cb, State: Initialized, Role: FOLLOWER
I20260809 23:16:05.116312 22597 consensus_queue.cc:260] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167a2393756f41ff9b5adc62ee35e9cb" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 45773 } } peers { permanent_uuid: "4e5335c78c964517be89c2ce0b3df4ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 41845 } } peers { permanent_uuid: "161b097116dc4d329fe8c7e52bc52ea3" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 45501 } }
I20260809 23:16:05.116590 22597 ts_tablet_manager.cc:1428] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 23:16:05.116722 22332 heartbeater.cc:503] Master 127.2.76.254:40105 was elected leader, sending a full tablet report...
Aug 09 23:16:05 dist-test-slave-7vjv krb5kdc[20925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317364, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
W20260809 23:16:05.270366 22564 tablet.cc:2369] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:16:05.273864 22333 tablet.cc:2369] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:16:05.306681 22096 tablet.cc:2369] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:16:05.356602 22604 raft_consensus.cc:482] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:16:05.356760 22604 raft_consensus.cc:504] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167a2393756f41ff9b5adc62ee35e9cb" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 45773 } } peers { permanent_uuid: "4e5335c78c964517be89c2ce0b3df4ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 41845 } } peers { permanent_uuid: "161b097116dc4d329fe8c7e52bc52ea3" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 45501 } }
I20260809 23:16:05.357161 22604 leader_election.cc:288] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 167a2393756f41ff9b5adc62ee35e9cb (127.2.76.194:45773), 4e5335c78c964517be89c2ce0b3df4ba (127.2.76.195:41845)
I20260809 23:16:05.361470 22589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.357595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:54141 (local address 127.2.76.195:41845)
0809 23:16:05.357635 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:05.357637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:05.358089 (+ 452us) server_negotiation.cc:407] Connection header received
0809 23:16:05.358196 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:05.358197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:05.358208 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:05.358236 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:05.358465 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.359155 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.360563 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.361057 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.361119 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:05.361128 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:05.361367 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 23:16:05.361405 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:16:05.361897 22608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.357418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:41845 (local address 127.2.76.193:54141)
0809 23:16:05.358031 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:05.358041 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:05.358118 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:05.358300 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:05.358301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:05.358302 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:05.358381 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:05.358383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.359344 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.359345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:05.360457 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:05.360460 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.361209 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.361209 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:05.361277 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:05.361281 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:16:05.361764 (+ 483us) client_negotiation.cc:240] Negotiation successful
0809 23:16:05.361806 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":82,"threads_started":1}
I20260809 23:16:05.362347 22485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c242fd679643968ad7e6bb71732909" candidate_uuid: "161b097116dc4d329fe8c7e52bc52ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5335c78c964517be89c2ce0b3df4ba" is_pre_election: true
I20260809 23:16:05.362495 22485 raft_consensus.cc:2454] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 161b097116dc4d329fe8c7e52bc52ea3 in term 0.
I20260809 23:16:05.362810 21913 leader_election.cc:302] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 161b097116dc4d329fe8c7e52bc52ea3, 4e5335c78c964517be89c2ce0b3df4ba; no voters:
I20260809 23:16:05.362943 22604 raft_consensus.cc:2781] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:16:05.362983 22604 raft_consensus.cc:482] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:16:05.363004 22604 raft_consensus.cc:3037] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:05.363078 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.357589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:59853 (local address 127.2.76.194:45773)
0809 23:16:05.357632 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:05.357634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:05.357752 (+ 118us) server_negotiation.cc:407] Connection header received
0809 23:16:05.357867 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:05.357868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:05.357889 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:05.357916 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:05.358445 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.359115 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.362346 (+ 3231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.362764 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.362802 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:05.362808 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:05.362989 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 23:16:05.363023 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:16:05.363276 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.357405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:45773 (local address 127.2.76.193:59853)
0809 23:16:05.357694 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:05.357703 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:05.357780 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:05.357983 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:05.357984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:05.358000 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:05.358078 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:05.358079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.360520 (+ 2441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.360521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:05.362252 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:05.362254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.362856 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.362856 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:05.362923 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:05.362925 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:16:05.363168 (+ 243us) client_negotiation.cc:240] Negotiation successful
0809 23:16:05.363201 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":85,"threads_started":1}
I20260809 23:16:05.363526 22258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c242fd679643968ad7e6bb71732909" candidate_uuid: "161b097116dc4d329fe8c7e52bc52ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "167a2393756f41ff9b5adc62ee35e9cb" is_pre_election: true
I20260809 23:16:05.363646 22258 raft_consensus.cc:2454] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 161b097116dc4d329fe8c7e52bc52ea3 in term 0.
I20260809 23:16:05.363888 22604 raft_consensus.cc:504] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167a2393756f41ff9b5adc62ee35e9cb" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 45773 } } peers { permanent_uuid: "4e5335c78c964517be89c2ce0b3df4ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 41845 } } peers { permanent_uuid: "161b097116dc4d329fe8c7e52bc52ea3" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 45501 } }
I20260809 23:16:05.364015 22604 leader_election.cc:288] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [CANDIDATE]: Term 1 election: Requested vote from peers 167a2393756f41ff9b5adc62ee35e9cb (127.2.76.194:45773), 4e5335c78c964517be89c2ce0b3df4ba (127.2.76.195:41845)
I20260809 23:16:05.364435 22258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c242fd679643968ad7e6bb71732909" candidate_uuid: "161b097116dc4d329fe8c7e52bc52ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "167a2393756f41ff9b5adc62ee35e9cb"
I20260809 23:16:05.364444 22485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c242fd679643968ad7e6bb71732909" candidate_uuid: "161b097116dc4d329fe8c7e52bc52ea3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e5335c78c964517be89c2ce0b3df4ba"
I20260809 23:16:05.364529 22258 raft_consensus.cc:3037] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:05.364529 22485 raft_consensus.cc:3037] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:05.365423 22258 raft_consensus.cc:2454] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 161b097116dc4d329fe8c7e52bc52ea3 in term 1.
I20260809 23:16:05.365427 22485 raft_consensus.cc:2454] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 161b097116dc4d329fe8c7e52bc52ea3 in term 1.
I20260809 23:16:05.365633 21912 leader_election.cc:302] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 161b097116dc4d329fe8c7e52bc52ea3, 167a2393756f41ff9b5adc62ee35e9cb; no voters:
I20260809 23:16:05.365787 22604 raft_consensus.cc:2781] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:16:05.365931 22604 raft_consensus.cc:686] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [term 1 LEADER]: Becoming Leader. State: Replica: 161b097116dc4d329fe8c7e52bc52ea3, State: Running, Role: LEADER
I20260809 23:16:05.366009 22604 consensus_queue.cc:237] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167a2393756f41ff9b5adc62ee35e9cb" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 45773 } } peers { permanent_uuid: "4e5335c78c964517be89c2ce0b3df4ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 41845 } } peers { permanent_uuid: "161b097116dc4d329fe8c7e52bc52ea3" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 45501 } }
I20260809 23:16:05.366765 21730 catalog_manager.cc:5495] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 161b097116dc4d329fe8c7e52bc52ea3 (127.2.76.193). New cstate: current_term: 1 leader_uuid: "161b097116dc4d329fe8c7e52bc52ea3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "167a2393756f41ff9b5adc62ee35e9cb" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 45773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e5335c78c964517be89c2ce0b3df4ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 41845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "161b097116dc4d329fe8c7e52bc52ea3" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 45501 } health_report { overall_health: HEALTHY } } }
Aug 09 23:16:05 dist-test-slave-7vjv krb5kdc[20925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317364, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:05.437356 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.434367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38106 (local address 127.2.76.193:45501)
0809 23:16:05.434425 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:05.434427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:05.434435 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:16:05.434474 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:05.434475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:05.434516 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:05.434585 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:16:05.434783 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.435412 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.436369 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:05.436617 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:05.436683 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:05.436927 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:16:05.437046 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:16:05.437108 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:05.437252 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 23:16:05.437270 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 23:16:05.439867 22485 raft_consensus.cc:1261] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba [term 1 FOLLOWER]: Refusing update from remote peer 161b097116dc4d329fe8c7e52bc52ea3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:16:05.440245 22611 consensus_queue.cc:1035] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e5335c78c964517be89c2ce0b3df4ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 41845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:16:05.442025 22258 raft_consensus.cc:1261] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb [term 1 FOLLOWER]: Refusing update from remote peer 161b097116dc4d329fe8c7e52bc52ea3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:16:05.442488 22611 consensus_queue.cc:1035] T 50c242fd679643968ad7e6bb71732909 P 161b097116dc4d329fe8c7e52bc52ea3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "167a2393756f41ff9b5adc62ee35e9cb" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 45773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:16:05.446378 22625 mvcc.cc:205] Tried to move back new op lower bound from 7316755928838197248 to 7316755928539447296. Current Snapshot: MvccSnapshot[applied={T|T < 7316755928838197248}]
I20260809 23:16:05.446583 22630 mvcc.cc:205] Tried to move back new op lower bound from 7316755928838197248 to 7316755928539447296. Current Snapshot: MvccSnapshot[applied={T|T < 7316755928838197248}]
I20260809 23:16:05.448598 21730 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:16:05.448679 21730 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:16:05.449453 21730 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2368643443a54b32a046f00daedae538: Sending DeleteTablet for 3 replicas of tablet 50c242fd679643968ad7e6bb71732909
I20260809 23:16:05.449946 22212 tablet_service.cc:1514] Processing DeleteTablet for tablet 50c242fd679643968ad7e6bb71732909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:16:05 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:36924
I20260809 23:16:05.450201 21978 tablet_service.cc:1514] Processing DeleteTablet for tablet 50c242fd679643968ad7e6bb71732909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:16:05 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:38096
I20260809 23:16:05.450655 22435 tablet_service.cc:1514] Processing DeleteTablet for tablet 50c242fd679643968ad7e6bb71732909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:16:05 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:36784
I20260809 23:16:05.451328 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 21873
I20260809 23:16:05.456470 22646 tablet_replica.cc:330] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba: stopping tablet replica
I20260809 23:16:05.456580 22646 raft_consensus.cc:2229] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:16:05.456650 22646 pending_rounds.cc:62] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba: Trying to abort 2 pending ops.
I20260809 23:16:05.456679 22646 pending_rounds.cc:68] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316755928838197248 op_type: WRITE_OP write_request { tablet_id: "50c242fd679643968ad7e6bb71732909" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\341\226\344\323\006\"8\n\ntest-admin\022*\n 066782826f6347fda3f4b63f36ed7c6d\020\001\030\001 \001(\001" signature: "<redacted>""r\r9\236\327p-\205\321\'e2\273&\031[\375\247\311\010\260\032\304\350U\351\306v\010\270\237\346\301]\262\270u\006\251K;P\233\034\337`w\037\267#\325}\017\233\327\274\006\221I\237\224\343q8" signing_key_seq_num: 0 } } request_id { client_id: "406bb4965d764200ac347ace063efa5c" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260809 23:16:05.458619 22646 pending_rounds.cc:68] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316755928539447296 op_type: NO_OP noop_request { }
I20260809 23:16:05.458688 22646 raft_consensus.cc:2866] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260809 23:16:05.458730 22646 raft_consensus.cc:2258] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 23:16:05.463231 22628 consensus_queue.cc:1159] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4e5335c78c964517be89c2ce0b3df4ba" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260809 23:16:05.463856 22646 ts_tablet_manager.cc:1905] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:16:05.466454 22646 ts_tablet_manager.cc:1918] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:16:05.466599 22646 log.cc:1196] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/50c242fd679643968ad7e6bb71732909
I20260809 23:16:05.467664 22646 ts_tablet_manager.cc:1939] T 50c242fd679643968ad7e6bb71732909 P 4e5335c78c964517be89c2ce0b3df4ba: Deleting consensus metadata
I20260809 23:16:05.468189 21707 catalog_manager.cc:4841] TS 4e5335c78c964517be89c2ce0b3df4ba (127.2.76.195:41845): tablet 50c242fd679643968ad7e6bb71732909 (table test-table [id=066782826f6347fda3f4b63f36ed7c6d]) successfully deleted
I20260809 23:16:05.483513 22644 tablet_replica.cc:330] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb: stopping tablet replica
I20260809 23:16:05.483669 22644 raft_consensus.cc:2229] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:16:05.483827 22644 raft_consensus.cc:2258] T 50c242fd679643968ad7e6bb71732909 P 167a2393756f41ff9b5adc62ee35e9cb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 23:16:05.485908 22363 connection.cc:535] server connection from 127.2.76.193:54141 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 23:16:05.486549 21706 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:45501 (error 108)
I20260809 23:16:05.486858 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 22108
I20260809 23:16:05.516717 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.516559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:45501 (local address 127.0.0.1:38108)
0809 23:16:05.516611 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:05.516662 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 23:16:05.517014 21706 catalog_manager.cc:4584] TS 161b097116dc4d329fe8c7e52bc52ea3 (127.2.76.193:45501): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 50c242fd679643968ad7e6bb71732909: Network error: Client connection negotiation failed: client connection to 127.2.76.193:45501: connect: Connection refused (error 111)
I20260809 23:16:05.523231 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 22339
I20260809 23:16:05.545238 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:05.545100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:45773 (local address 127.0.0.1:36934)
0809 23:16:05.545165 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:05.545196 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.194:45773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 23:16:05.547854 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 21667
I20260809 23:16:05.562503 2355 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 23:16:05.910038 2355 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 23:16:05.910130 2355 mini_ranger.cc:66] Stopping Ranger...
I20260809 23:16:06.265944 2355 mini_ranger.cc:68] Stopped Ranger
2026-08-09 23:16:06.266 UTC [20965] LOG: received smart shutdown request
2026-08-09 23:16:06.269 UTC [20965] LOG: background worker "logical replication launcher" (PID 20984) exited with exit code 1
2026-08-09 23:16:06.269 UTC [20976] LOG: shutting down
2026-08-09 23:16:06.409 UTC [20965] LOG: database system is shut down
2026-08-09T23:16:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (42911 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:16:06 dist-test-slave-7vjv krb5kdc[22662](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:16:06 dist-test-slave-7vjv krb5kdc[22662](info): set up 2 sockets
Aug 09 23:16:06 dist-test-slave-7vjv krb5kdc[22662](info): commencing operation
krb5kdc: starting...
W20260809 23:16:07.687765 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:16:07 dist-test-slave-7vjv krb5kdc[22662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:16:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:16:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.76.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.76.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/rangeradmin_127.2.76.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/rangeradmin_127.2.76.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.76.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.76.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/rangerlookup_127.2.76.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/rangerlookup_127.2.76.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.76.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.76.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/HTTP_127.2.76.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/HTTP_127.2.76.212@KRBTEST.COM.keytab.
I20260809 23:16:07.828547 2355 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskvJxnr9/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/postgres -l logfile start
2026-08-09 23:16:09.984 UTC [22718] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 23:16:09.984 UTC [22718] LOG: listening on IPv4 address "127.2.76.212", port 57019
2026-08-09 23:16:09.987 UTC [22718] LOG: listening on Unix socket "/tmp/.s.PGSQL.57019"
2026-08-09 23:16:10.007 UTC [22721] LOG: database system was shut down at 2026-08-09 23:16:08 UTC
2026-08-09 23:16:10.011 UTC [22718] LOG: database system is ready to accept connections
I20260809 23:16:11.079427 2355 mini_postgres.cc:96] Postgres bound to 57019
2026-08-09 23:16:11.086 UTC [22737] FATAL: database "slave" does not exist
127.2.76.212:57019 - accepting connections
I20260809 23:16:11.086826 2355 mini_ranger.cc:161] Starting Ranger...
I20260809 23:16:11.100739 2355 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 23:16:11.506179 2355 mini_ranger.cc:87] Created ranger Postgres database
I20260809 23:16:11.506264 2355 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-admin
2026-08-09 23:16:11,766 [I] DB FLAVOR :POSTGRES
2026-08-09 23:16:11,767 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 23:16:11,767 [I] Checking connection..
2026-08-09 23:16:11,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 23:16:12,000 [I] Checking connection passed.
2026-08-09 23:16:12,000 [I] --------- Verifying version history table ---------
2026-08-09 23:16:12,000 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:16:12,218 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 23:16:12,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:16:12,444 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 23:16:12,444 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 23:16:12,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 23:16:12.658 UTC [22837] WARNING: there is no transaction in progress
2026-08-09 23:16:12,666 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 23:16:12,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:16:12,904 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 23:16:12,905 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 23:16:12,905 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 23:16:13,131 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 23:16:13,371 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7vjv.c.gcp-upstream.internal','N') ;"
2026-08-09 23:16:13,601 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 23:16:13,810 [I] Table x_portal_user does not exist in database ranger
2026-08-09 23:16:13,810 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 23:16:14,016 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 23:16:14,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 23:16:14,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 23:16:14,507 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 23:16:14,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 23:16:15.324 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:15.339 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:15.353 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:15.366 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:15.379 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:15.394 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:15.441 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:15.446 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:15.458 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:15.467 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:15.475 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:15.488 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:15.496 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:15.501 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:15.507 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:15.513 UTC [23022] WARNING: there is no transaction in progress
2026-08-09 23:16:16,062 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 23:16:16,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 23:16:16,291 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 23:16:16,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 23:16:16,524 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 23:16:16,524 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 23:16:16,756 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 23:16:16,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 23:16:17,000 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 23:16:17,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7vjv.c.gcp-upstream.internal';"
2026-08-09 23:16:17,466 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 23:16:17,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 23:16:17,714 [I] DB_PATCHES have already been applied
I20260809 23:16:17.719823 2355 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-admin:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskvJxnr9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 23:16:17.719883 2355 mini_ranger.cc:193] Using host: 127.2.76.212
I20260809 23:16:17.723734 2355 mini_ranger.cc:227] Ranger admin URL: http://127.2.76.212:38271
Aug 09, 2026 11:16:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 11:16:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 11:16:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 11:16:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 11:16:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 11:16:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 11:16:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 23:16:19.049834 2355 mini_ranger.cc:160] Time spent starting Ranger: real 7.963s user 0.000s sys 0.009s
Aug 09, 2026 11:16:37 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 11:16:37 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 11:16:37 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 11:16:37 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 11:16:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 11:16:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 11:16:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 11:16:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 11:16:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 11:16:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 11:16:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 11:16:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 11:16:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 11:16:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 11:16:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 11:16:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 11:16:37 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 11:16:38 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 23:16:38.728176 2355 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.76.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.76.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/rangerkms_127.2.76.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.76.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/rangerkms_127.2.76.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.76.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.76.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.76.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/HTTP_127.2.76.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.76.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/HTTP_127.2.76.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 23:16:38 dist-test-slave-7vjv krb5kdc[22662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317398, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 23:16:38.805359 2355 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 23:16:38.814100 2355 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 23:16:39.584219 2355 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 23:16:39.584306 2355 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-kms
I20260809 23:16:39.584324 2355 mini_ranger_kms.cc:227] Using postgres at 127.2.76.212:57019
2026-08-09 23:16:39,756 [I] DB FLAVOR :POSTGRES
2026-08-09 23:16:39,756 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 23:16:39,756 [I] Checking connection
2026-08-09 23:16:39,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 23:16:39,963 [I] connection success
2026-08-09 23:16:39,964 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 23:16:39,964 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 23:16:39,964 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 23:16:40,188 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 23:16:40,188 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 23:16:40,188 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 23:16:40,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.76.212:57019/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 23:16:40,435 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 23:16:40.857322 2355 mini_ranger_kms.cc:310] Created kms service
I20260809 23:16:41.033694 2355 mini_ranger_kms.cc:326] Created kudu user
I20260809 23:16:41.073290 2355 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 23:16:41 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317398, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:41.461760 2355 mini_ranger_kms.cc:384] Added ranger policy
I20260809 23:16:41.461894 2355 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-kms:/tmp/dist-test-taskvJxnr9/build/release/bin/postgresql.jar:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskvJxnr9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 23:16:41.461934 2355 mini_ranger_kms.cc:240] Using host: 127.2.76.212
I20260809 23:16:41.465945 2355 mini_ranger_kms.cc:276] Ranger KMS PID: 23323
I20260809 23:16:41.466023 2355 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.76.212:53295
Aug 09, 2026 11:16:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 11:16:42 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 11:16:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 11:16:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 11:16:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 11:16:42 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 23:16:43 dist-test-slave-7vjv krb5kdc[22662](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786317403, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.76.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 23:16:43 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317403, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.76.212@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:43.982465 2355 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.177s user 0.002s sys 0.156s
I20260809 23:16:43.982580 2355 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 23:16:43.982637 2355 mini_ranger_kms.cc:401] 127.2.76.212:53295/kms/v1/keys
Aug 09 23:16:44 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317403, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.76.212@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
Aug 09 23:16:46 dist-test-slave-7vjv krb5kdc[22662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:16:46.327302 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:41229
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:41229
--ranger_config_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.76.212:53295/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:46.406940 23401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:46.407123 23401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:46.407143 23401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:46.408234 23401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:16:46.408272 23401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:46.408285 23401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:46.408296 23401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:16:46.408308 23401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:16:46.409745 23401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.76.212:53295/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:41229
--ranger_config_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:41229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:46.409953 23401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:46.410205 23401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:46.414680 23417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:46.414688 23414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:46.414696 23415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:46.414906 23401 server_base.cc:1034] running on GCE node
I20260809 23:16:46.415365 23401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:46.415669 23401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:46.416808 23401 hybrid_clock.cc:648] HybridClock initialized: now 1786317406416790 us; error 33 us; skew 500 ppm
Aug 09 23:16:46 dist-test-slave-7vjv krb5kdc[22662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317406, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:46.419338 23401 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:16:46.419874 23401 webserver.cc:460] Webserver started at http://127.2.76.254:35257/ using document root <none> and password file <none>
I20260809 23:16:46.420149 23401 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:46.420209 23401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:46.420352 23401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:16:46 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317406, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:46.463754 23401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "d717204bbbbf435f83c832078bcbcd6f"
format_stamp: "Formatted at 2026-08-09 23:16:46 on dist-test-slave-7vjv"
server_key: "878df6e4c0cb3048797905359cff9d2b"
server_key_iv: "c852084271bbffc58a6b62d0a650e565"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:46.464316 23401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "d717204bbbbf435f83c832078bcbcd6f"
format_stamp: "Formatted at 2026-08-09 23:16:46 on dist-test-slave-7vjv"
server_key: "878df6e4c0cb3048797905359cff9d2b"
server_key_iv: "c852084271bbffc58a6b62d0a650e565"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:46.465755 23401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:16:46.475276 23427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:46.475445 23401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:46.475533 23401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "d717204bbbbf435f83c832078bcbcd6f"
format_stamp: "Formatted at 2026-08-09 23:16:46 on dist-test-slave-7vjv"
server_key: "878df6e4c0cb3048797905359cff9d2b"
server_key_iv: "c852084271bbffc58a6b62d0a650e565"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:46.475611 23401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:46.502797 23401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:46.504551 23401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:46.504690 23401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:46.508534 23401 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:41229
I20260809 23:16:46.508576 23522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:41229 every 8 connection(s)
I20260809 23:16:46.508896 23401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:16:46.509485 23526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:46.511786 23526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f: Bootstrap starting.
I20260809 23:16:46.512344 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 23401
I20260809 23:16:46.512439 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:16:46.512439 23526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:46.512624 2355 external_mini_cluster.cc:1428] Setting key ada7dcceeae11a6253532f1fb6d5b701
I20260809 23:16:46.512784 23526 log.cc:824] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:46.513657 23526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f: No bootstrap required, opened a new log
I20260809 23:16:46.515794 23526 raft_consensus.cc:348] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717204bbbbf435f83c832078bcbcd6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 41229 } }
I20260809 23:16:46.515955 23526 raft_consensus.cc:374] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:46.515985 23526 raft_consensus.cc:729] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d717204bbbbf435f83c832078bcbcd6f, State: Initialized, Role: FOLLOWER
I20260809 23:16:46.516119 23526 consensus_queue.cc:260] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717204bbbbf435f83c832078bcbcd6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 41229 } }
I20260809 23:16:46.516196 23526 raft_consensus.cc:388] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:16:46.516232 23526 raft_consensus.cc:482] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:16:46.516283 23526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:46.516762 23526 raft_consensus.cc:504] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717204bbbbf435f83c832078bcbcd6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 41229 } }
I20260809 23:16:46.516875 23526 leader_election.cc:302] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d717204bbbbf435f83c832078bcbcd6f; no voters:
I20260809 23:16:46.517035 23526 leader_election.cc:288] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:16:46.517105 23533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:16:46.517262 23526 sys_catalog.cc:564] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:16:46.517292 23533 raft_consensus.cc:686] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 1 LEADER]: Becoming Leader. State: Replica: d717204bbbbf435f83c832078bcbcd6f, State: Running, Role: LEADER
I20260809 23:16:46.517477 23533 consensus_queue.cc:237] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717204bbbbf435f83c832078bcbcd6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 41229 } }
I20260809 23:16:46.518183 23536 sys_catalog.cc:455] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d717204bbbbf435f83c832078bcbcd6f. Latest consensus state: current_term: 1 leader_uuid: "d717204bbbbf435f83c832078bcbcd6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717204bbbbf435f83c832078bcbcd6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 41229 } } }
I20260809 23:16:46.518282 23536 sys_catalog.cc:458] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:46.518664 23534 sys_catalog.cc:455] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d717204bbbbf435f83c832078bcbcd6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717204bbbbf435f83c832078bcbcd6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 41229 } } }
I20260809 23:16:46.518710 23526 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 23:16:46.518716 23534 sys_catalog.cc:458] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:46.519069 23551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 23:16:46 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:46.522164 23551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:16:46.525532 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:46.513086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.2.76.254:41229)
0809 23:16:46.513248 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:46.513254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:46.513354 (+ 100us) server_negotiation.cc:407] Connection header received
0809 23:16:46.513895 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:46.513904 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:46.513957 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:46.514092 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:46.514761 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:46.516157 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:46.516945 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:46.517525 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:46.517623 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:46.520994 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:46.521056 (+ 62us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:46.521060 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:46.521170 (+ 110us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:46.524054 (+ 2884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:46.524449 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:46.524451 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:46.524451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:46.524554 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:46.524728 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:46.524729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:46.524729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:46.524950 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:46.525108 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:46.525177 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 23:16:46.525307 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":63,"threads_started":1}
I20260809 23:16:46.528497 23551 catalog_manager.cc:1347] Generated new cluster ID: e483213d99cd4ec4bf531ad1ab93244c
I20260809 23:16:46.528578 23551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:16:46.536724 23551 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:16:46.537314 23551 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:16:46.543953 23551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f: Generated new TSK 0
I20260809 23:16:46.544111 23551 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:16:47.139173 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.76.212:53295/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:41229
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:47.235500 23602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:47.235689 23602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:47.235713 23602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:47.237188 23602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:47.237241 23602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:47.237267 23602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:16:47.239048 23602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.76.212:53295/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:47.239291 23602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:47.239593 23602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:47.245404 23609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:47.245546 23602 server_base.cc:1034] running on GCE node
W20260809 23:16:47.245446 23610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:47.245433 23612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:47.245898 23602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:47.246163 23602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:47.247345 23602 hybrid_clock.cc:648] HybridClock initialized: now 1786317407247324 us; error 42 us; skew 500 ppm
Aug 09 23:16:47 dist-test-slave-7vjv krb5kdc[22662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:47.250274 23602 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:16:47.250654 23602 webserver.cc:460] Webserver started at http://127.2.76.193:33241/ using document root <none> and password file <none>
I20260809 23:16:47.250875 23602 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:47.250938 23602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:47.251073 23602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:16:47 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:47.262372 23602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "c647355fe82f4cad9d83a7d8a3717420"
format_stamp: "Formatted at 2026-08-09 23:16:47 on dist-test-slave-7vjv"
server_key: "e425f1f66fe1636539b141822b7ca6ba"
server_key_iv: "304f50b0c4536bbcede64216647d8c2e"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:47.262774 23602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "c647355fe82f4cad9d83a7d8a3717420"
format_stamp: "Formatted at 2026-08-09 23:16:47 on dist-test-slave-7vjv"
server_key: "e425f1f66fe1636539b141822b7ca6ba"
server_key_iv: "304f50b0c4536bbcede64216647d8c2e"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:47.264449 23602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:16:47.276180 23627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:47.276348 23602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:16:47.276424 23602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "c647355fe82f4cad9d83a7d8a3717420"
format_stamp: "Formatted at 2026-08-09 23:16:47 on dist-test-slave-7vjv"
server_key: "e425f1f66fe1636539b141822b7ca6ba"
server_key_iv: "304f50b0c4536bbcede64216647d8c2e"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:47.276484 23602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:47.287950 23602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:47.289969 23602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:47.290112 23602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:47.290263 23602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:47.290581 23602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:47.290622 23602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:47.290647 23602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:47.290688 23602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:47.296914 23602 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:46055
I20260809 23:16:47.297358 23602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:16:47 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:47.304702 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 23602
I20260809 23:16:47.304802 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:16:47.305001 2355 external_mini_cluster.cc:1428] Setting key ce0fdbdc45cb494f139b6ba801568c90
I20260809 23:16:47.307351 23834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:47.298078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:41229 (local address 127.2.76.193:35845)
0809 23:16:47.298258 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:47.298298 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:47.298865 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:47.299234 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:47.299237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:47.299452 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:47.299689 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:47.299695 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:47.300641 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:47.300643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:47.301707 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:47.301709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:47.302179 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:47.302180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:47.302304 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:47.302770 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:47.302779 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:47.304178 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:47.306018 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:47.306024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:47.306026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:47.306261 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:47.306461 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:47.306462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:47.306462 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:47.306574 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:47.306813 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:47.306818 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:47.306944 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 23:16:47.307034 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 23:16:47.307155 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260809 23:16:47.308027 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:47.298114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:35845 (local address 127.2.76.254:41229)
0809 23:16:47.299034 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:47.299036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:47.299045 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:16:47.299070 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:47.299071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:47.299082 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:47.299777 (+ 695us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:47.299820 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:47.300547 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:47.301804 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:47.302084 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:47.302151 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:47.304274 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:47.304275 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:47.304276 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:47.304282 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:47.305936 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:47.306343 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:47.306344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:47.306345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:47.306401 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:47.306633 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:47.306634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:47.306634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:47.306743 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:47.306792 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:47.307869 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0809 23:16:47.307945 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":890,"thread_start_us":74,"threads_started":1}
I20260809 23:16:47.308502 23830 heartbeater.cc:347] Connected to a master server at 127.2.76.254:41229
I20260809 23:16:47.308574 23830 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:47.308718 23830 heartbeater.cc:511] Master 127.2.76.254:41229 requested a full tablet report, sending...
I20260809 23:16:47.309196 23466 ts_manager.cc:194] Registered new tserver with Master: c647355fe82f4cad9d83a7d8a3717420 (127.2.76.193:46055)
I20260809 23:16:47.310007 23466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:35845
I20260809 23:16:47.298696 23829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:46055 every 8 connection(s)
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:16:47.334532 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.76.212:53295/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:41229
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:16:47 dist-test-slave-7vjv krb5kdc[22662](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786317407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 23:16:47.446708 23842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:47.447098 23842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:47.447184 23842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:47.448792 23842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:47.448844 23842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:47.448877 23842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:16:47.450816 23842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.76.212:53295/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:47.451239 23842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:47.451514 23842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:47.457335 23854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:47.457366 23853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:47.457638 23856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:47.458432 23842 server_base.cc:1034] running on GCE node
I20260809 23:16:47.458619 23842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:47.458834 23842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:47.460069 23842 hybrid_clock.cc:648] HybridClock initialized: now 1786317407460051 us; error 36 us; skew 500 ppm
Aug 09 23:16:47 dist-test-slave-7vjv krb5kdc[22662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:47.462906 23842 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:16:47.463332 23842 webserver.cc:460] Webserver started at http://127.2.76.194:45659/ using document root <none> and password file <none>
I20260809 23:16:47.463581 23842 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:47.463632 23842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:47.463792 23842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:16:47 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:47.476395 23842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "b750e93d33e44b3892c6365fcbdee4b6"
format_stamp: "Formatted at 2026-08-09 23:16:47 on dist-test-slave-7vjv"
server_key: "fe4ad0775d70735c87173af5a0c1ec1d"
server_key_iv: "29420586597295416916b7f5733e3320"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:47.476907 23842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "b750e93d33e44b3892c6365fcbdee4b6"
format_stamp: "Formatted at 2026-08-09 23:16:47 on dist-test-slave-7vjv"
server_key: "fe4ad0775d70735c87173af5a0c1ec1d"
server_key_iv: "29420586597295416916b7f5733e3320"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:47.480508 23842 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260809 23:16:47.491868 23873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:47.492041 23842 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:16:47.492162 23842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "b750e93d33e44b3892c6365fcbdee4b6"
format_stamp: "Formatted at 2026-08-09 23:16:47 on dist-test-slave-7vjv"
server_key: "fe4ad0775d70735c87173af5a0c1ec1d"
server_key_iv: "29420586597295416916b7f5733e3320"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:47.492246 23842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:47.506639 23842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:47.511252 23842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:47.511433 23842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:47.511626 23842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:47.511960 23842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:47.512039 23842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:47.512120 23842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:47.512176 23842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:47.518689 23842 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:43461
I20260809 23:16:47.519101 23842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:16:47.521773 24045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:43461 every 8 connection(s)
I20260809 23:16:47.525085 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 23842
I20260809 23:16:47.525200 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:16:47.525388 2355 external_mini_cluster.cc:1428] Setting key d460fa5d775a5976ad3d10df8aebc637
Aug 09 23:16:47 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:47.533041 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:47.522373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:50759 (local address 127.2.76.254:41229)
0809 23:16:47.522416 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:47.522419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:47.522636 (+ 217us) server_negotiation.cc:407] Connection header received
0809 23:16:47.523300 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:47.523301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:47.523315 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:47.523356 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:47.524140 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:47.524876 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:47.527157 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:47.527460 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:47.527500 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:47.530003 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:47.530005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:47.530005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:47.530012 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:47.531590 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:47.532025 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:47.532026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:47.532026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:47.532085 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:47.532388 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:47.532389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:47.532390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:47.532525 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:47.532595 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:47.532878 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 23:16:47.532962 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:16:47.533152 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:47.522374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:41229 (local address 127.2.76.194:50759)
0809 23:16:47.522527 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:47.522563 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:47.523177 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:47.523530 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:47.523534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:47.523755 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:47.524005 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:47.524011 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:47.525943 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:47.525943 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:47.527054 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:47.527057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:47.527599 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:47.527599 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:47.527720 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:47.528438 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:47.528446 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:47.529904 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:47.531681 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:47.531687 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:47.531689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:47.531942 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:47.532179 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:47.532180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:47.532180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:47.532305 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:47.532597 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:47.532602 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:47.532731 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 23:16:47.532832 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 23:16:47.532961 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
I20260809 23:16:47.533931 24046 heartbeater.cc:347] Connected to a master server at 127.2.76.254:41229
I20260809 23:16:47.534021 24046 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:47.534185 24046 heartbeater.cc:511] Master 127.2.76.254:41229 requested a full tablet report, sending...
I20260809 23:16:47.534550 23466 ts_manager.cc:194] Registered new tserver with Master: b750e93d33e44b3892c6365fcbdee4b6 (127.2.76.194:43461)
I20260809 23:16:47.534976 23466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:50759
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:16:47.566794 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.76.212:53295/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:41229
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:47.672138 24052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:47.672441 24052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:47.672525 24052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:47.673638 24052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:47.673691 24052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:47.673728 24052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:16:47.675277 24052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.76.212:53295/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:47.675613 24052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:47.675807 24052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:47.680904 24058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:47.681372 24063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:47.680948 24059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:47.683144 24052 server_base.cc:1034] running on GCE node
I20260809 23:16:47.683344 24052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:47.683594 24052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:47.684798 24052 hybrid_clock.cc:648] HybridClock initialized: now 1786317407684770 us; error 43 us; skew 500 ppm
Aug 09 23:16:47 dist-test-slave-7vjv krb5kdc[22662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:47.687702 24052 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:16:47.688197 24052 webserver.cc:460] Webserver started at http://127.2.76.195:35173/ using document root <none> and password file <none>
I20260809 23:16:47.688450 24052 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:47.688509 24052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:47.688689 24052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:16:47 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:47.707026 24052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "b8aea4dce8b7435786d9dbeefd11e032"
format_stamp: "Formatted at 2026-08-09 23:16:47 on dist-test-slave-7vjv"
server_key: "8310c3e7c64eab3120c6eb52813a5304"
server_key_iv: "0b989dd00958522f8e2677af71cb666b"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:47.707396 24052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "b8aea4dce8b7435786d9dbeefd11e032"
format_stamp: "Formatted at 2026-08-09 23:16:47 on dist-test-slave-7vjv"
server_key: "8310c3e7c64eab3120c6eb52813a5304"
server_key_iv: "0b989dd00958522f8e2677af71cb666b"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:47.711421 24052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260809 23:16:47.720798 24080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:47.721051 24052 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:16:47.721220 24052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "b8aea4dce8b7435786d9dbeefd11e032"
format_stamp: "Formatted at 2026-08-09 23:16:47 on dist-test-slave-7vjv"
server_key: "8310c3e7c64eab3120c6eb52813a5304"
server_key_iv: "0b989dd00958522f8e2677af71cb666b"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:47.721439 24052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:47.755367 24052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:47.757777 24052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:47.757972 24052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:47.758219 24052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:47.758597 24052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:47.758673 24052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:47.758734 24052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:47.758780 24052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:47.768992 24052 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:34573
I20260809 23:16:47.769420 24052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:16:47.771577 24270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:34573 every 8 connection(s)
I20260809 23:16:47.778749 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 24052
I20260809 23:16:47.778892 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:16:47.779106 2355 external_mini_cluster.cc:1428] Setting key a93ae9cdec64811b0aecc178ab10792e
Aug 09 23:16:47 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:47.795562 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:47.772505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:46947 (local address 127.2.76.254:41229)
0809 23:16:47.772553 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:47.772555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:47.777836 (+ 5281us) server_negotiation.cc:407] Connection header received
0809 23:16:47.778423 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:47.778425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:47.778439 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:47.778503 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:47.779303 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:47.779982 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:47.781232 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:47.781506 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:47.781554 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:47.786600 (+ 5046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:47.786601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:47.786602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:47.786609 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:47.793693 (+ 7084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:47.794249 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:47.794250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:47.794251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:47.794332 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:47.794668 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:47.794669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:47.794670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:47.794822 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:47.794886 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:47.795327 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 23:16:47.795424 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:16:47.796394 24278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:47.772216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:41229 (local address 127.2.76.195:46947)
0809 23:16:47.777684 (+ 5468us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:47.777727 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:47.778303 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:47.778570 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:47.778575 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:47.778887 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:47.779168 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:47.779175 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:47.780129 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:47.780129 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:47.781116 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:47.781120 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:47.781687 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:47.781688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:47.781818 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:47.782370 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:47.782381 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:47.786487 (+ 4106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:47.793796 (+ 7309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:47.793804 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:47.793807 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:47.794146 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:47.794421 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:47.794422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:47.794422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:47.794578 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:47.794926 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:47.794934 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:47.795126 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 23:16:47.795981 (+ 855us) client_negotiation.cc:240] Negotiation successful
0809 23:16:47.796163 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5424,"thread_start_us":89,"threads_started":1}
I20260809 23:16:47.798207 24271 heartbeater.cc:347] Connected to a master server at 127.2.76.254:41229
I20260809 23:16:47.798328 24271 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:47.798559 24271 heartbeater.cc:511] Master 127.2.76.254:41229 requested a full tablet report, sending...
I20260809 23:16:47.799592 23466 ts_manager.cc:194] Registered new tserver with Master: b8aea4dce8b7435786d9dbeefd11e032 (127.2.76.195:34573)
I20260809 23:16:47.800047 23466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:46947
I20260809 23:16:47.807153 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:16:47.807920 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 23602
I20260809 23:16:47.819252 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 23842
I20260809 23:16:47.827698 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 24052
I20260809 23:16:47.841300 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 23401
I20260809 23:16:47.849517 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:41229
--webserver_interface=127.2.76.254
--webserver_port=35257
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:41229
--ranger_config_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.76.212:53295/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.76.212:53295/kms with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:47.921577 24279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:47.921777 24279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:47.921813 24279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:47.922921 24279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:16:47.923002 24279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:47.923027 24279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:47.923063 24279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:16:47.923102 24279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:16:47.924461 24279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.76.212:53295/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:41229
--ranger_config_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:41229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=35257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:47.924685 24279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:47.924863 24279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:47.929096 24288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:47.929118 24279 server_base.cc:1034] running on GCE node
W20260809 23:16:47.929117 24291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:47.929101 24289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:47.929548 24279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:47.929806 24279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:47.930955 24279 hybrid_clock.cc:648] HybridClock initialized: now 1786317407930951 us; error 33 us; skew 500 ppm
Aug 09 23:16:47 dist-test-slave-7vjv krb5kdc[22662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:47.933025 24279 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:16:47.933368 24279 webserver.cc:460] Webserver started at http://127.2.76.254:35257/ using document root <none> and password file <none>
I20260809 23:16:47.933555 24279 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:47.933621 24279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 23:16:47 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:48.010623 24279 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:16:48.011374 24302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:48.011525 24279 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:16:48.011624 24279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "d717204bbbbf435f83c832078bcbcd6f"
format_stamp: "Formatted at 2026-08-09 23:16:46 on dist-test-slave-7vjv"
server_key: "878df6e4c0cb3048797905359cff9d2b"
server_key_iv: "c852084271bbffc58a6b62d0a650e565"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:48.011943 24279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:48.026724 24279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:48.028187 24279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:48.028319 24279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:48.031978 24279 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:41229
I20260809 23:16:48.032012 24382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:41229 every 8 connection(s)
I20260809 23:16:48.032366 24279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:16:48.033412 24383 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:16:48.034065 24383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f: Bootstrap starting.
I20260809 23:16:48.035696 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 24279
I20260809 23:16:48.036010 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.76.212:53295/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:46055
--local_ip_for_outbound_sockets=127.2.76.193
--tserver_master_addrs=127.2.76.254:41229
--webserver_port=33241
--webserver_interface=127.2.76.193
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.76.212:53295/kms with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:16:48.042155 24383 log.cc:824] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:48.043524 24383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:16:48.043790 24383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f: Bootstrap complete.
I20260809 23:16:48.045897 24383 raft_consensus.cc:348] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717204bbbbf435f83c832078bcbcd6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 41229 } }
I20260809 23:16:48.046119 24383 raft_consensus.cc:729] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d717204bbbbf435f83c832078bcbcd6f, State: Initialized, Role: FOLLOWER
I20260809 23:16:48.046281 24383 consensus_queue.cc:260] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717204bbbbf435f83c832078bcbcd6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 41229 } }
I20260809 23:16:48.046360 24383 raft_consensus.cc:388] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:16:48.046404 24383 raft_consensus.cc:482] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:16:48.046445 24383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 1 FOLLOWER]: Advancing to term 2
I20260809 23:16:48.047366 24383 raft_consensus.cc:504] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717204bbbbf435f83c832078bcbcd6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 41229 } }
I20260809 23:16:48.047483 24383 leader_election.cc:302] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d717204bbbbf435f83c832078bcbcd6f; no voters:
I20260809 23:16:48.047652 24383 leader_election.cc:288] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 23:16:48.047698 24387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 23:16:48.047843 24387 raft_consensus.cc:686] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [term 2 LEADER]: Becoming Leader. State: Replica: d717204bbbbf435f83c832078bcbcd6f, State: Running, Role: LEADER
I20260809 23:16:48.047902 24383 sys_catalog.cc:564] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:16:48.047941 24387 consensus_queue.cc:237] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717204bbbbf435f83c832078bcbcd6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 41229 } }
I20260809 23:16:48.048244 24388 sys_catalog.cc:455] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d717204bbbbf435f83c832078bcbcd6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717204bbbbf435f83c832078bcbcd6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 41229 } } }
I20260809 23:16:48.048243 24387 sys_catalog.cc:455] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d717204bbbbf435f83c832078bcbcd6f. Latest consensus state: current_term: 2 leader_uuid: "d717204bbbbf435f83c832078bcbcd6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d717204bbbbf435f83c832078bcbcd6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 41229 } } }
I20260809 23:16:48.048454 24388 sys_catalog.cc:458] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:48.048471 24387 sys_catalog.cc:458] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:48.049094 24399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:16:48.049461 24383 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 23:16:48.049546 24399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:16:48.049677 24399 catalog_manager.cc:1259] Loaded cluster ID: e483213d99cd4ec4bf531ad1ab93244c
I20260809 23:16:48.049710 24399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:16:48.053452 24399 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:16:48.053922 24399 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f: Loaded TSK: 0
I20260809 23:16:48.054164 24399 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 23:16:48.115883 24385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:48.116120 24385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:48.116163 24385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:48.117587 24385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:48.117641 24385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:48.117681 24385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:16:48.119544 24385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.76.212:53295/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:46055
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=33241
--tserver_master_addrs=127.2.76.254:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:48.119748 24385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:48.119961 24385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:48.125283 24446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:48.125396 24449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:48.125955 24454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:48.126188 24385 server_base.cc:1034] running on GCE node
I20260809 23:16:48.126370 24385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:48.126605 24385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:48.127760 24385 hybrid_clock.cc:648] HybridClock initialized: now 1786317408127734 us; error 42 us; skew 500 ppm
Aug 09 23:16:48 dist-test-slave-7vjv krb5kdc[22662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317408, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:48.131911 24385 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:16:48.132416 24385 webserver.cc:460] Webserver started at http://127.2.76.193:33241/ using document root <none> and password file <none>
I20260809 23:16:48.132649 24385 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:48.132730 24385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 23:16:48 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317408, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:48.160432 24385 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.001s sys 0.000s
I20260809 23:16:48.161490 24470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:48.161690 24385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:16:48.161767 24385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "c647355fe82f4cad9d83a7d8a3717420"
format_stamp: "Formatted at 2026-08-09 23:16:47 on dist-test-slave-7vjv"
server_key: "e425f1f66fe1636539b141822b7ca6ba"
server_key_iv: "304f50b0c4536bbcede64216647d8c2e"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:48.162160 24385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:48.171450 24385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:48.173550 24385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:48.173704 24385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:48.173878 24385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:48.174192 24385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:48.174225 24385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:48.174249 24385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:48.174264 24385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:48.180207 24385 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:46055
I20260809 23:16:48.180622 24385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:16:48.182438 24656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:46055 every 8 connection(s)
I20260809 23:16:48.184607 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 24385
I20260809 23:16:48.184896 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.76.212:53295/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:43461
--local_ip_for_outbound_sockets=127.2.76.194
--tserver_master_addrs=127.2.76.254:41229
--webserver_port=45659
--webserver_interface=127.2.76.194
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.76.212:53295/kms with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:16:48 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317408, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:48.194389 24659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:48.182982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:41229 (local address 127.2.76.193:59157)
0809 23:16:48.183299 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:48.183326 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:48.183752 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:48.184198 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:48.184201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:48.184359 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:48.184579 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:48.184586 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:48.186528 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:48.186529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:48.187810 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:48.187813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:48.188606 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:48.188608 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:48.188732 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:48.189222 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:48.189231 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:48.190689 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:48.192766 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:48.192773 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:48.192775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:48.193037 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:48.193301 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:48.193301 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:48.193302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:48.193422 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:48.193853 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:48.193859 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:48.193986 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 23:16:48.194072 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 23:16:48.194202 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":85,"threads_started":1}
I20260809 23:16:48.195281 24660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:48.183051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:59157 (local address 127.2.76.254:41229)
0809 23:16:48.183233 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:48.183245 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:48.183416 (+ 171us) server_negotiation.cc:407] Connection header received
0809 23:16:48.183968 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:48.183972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:48.184010 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:48.184150 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:48.184717 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:48.186410 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:48.187895 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:48.188471 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:48.188966 (+ 495us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:48.190821 (+ 1855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:48.190825 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:48.190826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:48.190836 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:48.192675 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:48.193128 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:48.193131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:48.193131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:48.193227 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:48.193505 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:48.193505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:48.193505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:48.193764 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:48.194884 (+ 1120us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:48.194961 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 23:16:48.195067 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
I20260809 23:16:48.196048 24657 heartbeater.cc:347] Connected to a master server at 127.2.76.254:41229
I20260809 23:16:48.196151 24657 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:48.196305 24657 heartbeater.cc:511] Master 127.2.76.254:41229 requested a full tablet report, sending...
W20260809 23:16:48.197089 24657 heartbeater.cc:644] Failed to heartbeat to 127.2.76.254:41229 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 23:16:48.197165 24657 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:48.197227 24657 heartbeater.cc:511] Master 127.2.76.254:41229 requested a full tablet report, sending...
I20260809 23:16:48.197535 24657 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:48.197606 24657 heartbeater.cc:511] Master 127.2.76.254:41229 requested a full tablet report, sending...
W20260809 23:16:48.197875 24657 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 23:16:48.292400 24661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:48.292625 24661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:48.292660 24661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:48.293919 24661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:48.293962 24661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:48.293984 24661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:16:48.295408 24661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.76.212:53295/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:43461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=45659
--tserver_master_addrs=127.2.76.254:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:48.295624 24661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:48.295804 24661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:48.301462 24669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:48.301852 24661 server_base.cc:1034] running on GCE node
W20260809 23:16:48.301538 24670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:48.305900 24673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:48.306150 24661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:48.306403 24661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:48.307588 24661 hybrid_clock.cc:648] HybridClock initialized: now 1786317408307582 us; error 30 us; skew 500 ppm
Aug 09 23:16:48 dist-test-slave-7vjv krb5kdc[22662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317408, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:48.310650 24661 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:16:48.311048 24661 webserver.cc:460] Webserver started at http://127.2.76.194:45659/ using document root <none> and password file <none>
I20260809 23:16:48.311276 24661 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:48.311420 24661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 23:16:48 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317408, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:48.327450 24661 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:16:48.328523 24683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:48.332505 24661 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260809 23:16:48.332592 24661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "b750e93d33e44b3892c6365fcbdee4b6"
format_stamp: "Formatted at 2026-08-09 23:16:47 on dist-test-slave-7vjv"
server_key: "fe4ad0775d70735c87173af5a0c1ec1d"
server_key_iv: "29420586597295416916b7f5733e3320"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:48.333005 24661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:48.349049 24661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:48.350868 24661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:48.351015 24661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:48.351228 24661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:48.351550 24661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:48.351598 24661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:48.351655 24661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:48.351679 24661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:48.357622 24661 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:43461
I20260809 23:16:48.358103 24661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:16:48.362701 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 24661
I20260809 23:16:48.362972 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.76.212:53295/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:34573
--local_ip_for_outbound_sockets=127.2.76.195
--tserver_master_addrs=127.2.76.254:41229
--webserver_port=35173
--webserver_interface=127.2.76.195
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.76.212:53295/kms with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:16:48.368710 24863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:43461 every 8 connection(s)
Aug 09 23:16:48 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317408, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:48.377864 24660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:48.369294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:43315 (local address 127.2.76.254:41229)
0809 23:16:48.369338 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:48.369340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:48.369590 (+ 250us) server_negotiation.cc:407] Connection header received
0809 23:16:48.370035 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:48.370037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:48.370051 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:48.370104 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:48.371701 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:48.372197 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:48.373059 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:48.373235 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:48.373298 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:48.375050 (+ 1752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:48.375051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:48.375052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:48.375059 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:48.376424 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:48.376868 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:48.376869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:48.376869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:48.376925 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:48.377185 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:48.377186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:48.377186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:48.377314 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:48.377355 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:48.377693 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 23:16:48.377780 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:16:48.378041 24872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:48.369303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:41229 (local address 127.2.76.194:43315)
0809 23:16:48.369461 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:48.369510 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:48.369939 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:48.371278 (+ 1339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:48.371282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:48.371429 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:48.371606 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:48.371610 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:48.372291 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:48.372292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:48.372974 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:48.372976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:48.373336 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:48.373337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:48.373429 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:48.373867 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:48.373874 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:48.374965 (+ 1091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:48.376515 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:48.376521 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:48.376523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:48.376786 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:48.376990 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:48.376990 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:48.376991 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:48.377116 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:48.377401 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:48.377407 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:48.377552 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 23:16:48.377671 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 23:16:48.377799 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260809 23:16:48.378509 24866 heartbeater.cc:347] Connected to a master server at 127.2.76.254:41229
I20260809 23:16:48.378594 24866 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:48.378760 24866 heartbeater.cc:511] Master 127.2.76.254:41229 requested a full tablet report, sending...
W20260809 23:16:48.379330 24866 heartbeater.cc:644] Failed to heartbeat to 127.2.76.254:41229 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 23:16:48.379402 24866 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:48.379469 24866 heartbeater.cc:511] Master 127.2.76.254:41229 requested a full tablet report, sending...
I20260809 23:16:48.379763 24866 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:48.379837 24866 heartbeater.cc:511] Master 127.2.76.254:41229 requested a full tablet report, sending...
W20260809 23:16:48.380128 24866 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 23:16:48.489315 24871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:48.489549 24871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:48.489590 24871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:48.491421 24871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:48.491521 24871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:48.491592 24871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:16:48.493216 24871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.76.212:53295/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:34573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=35173
--tserver_master_addrs=127.2.76.254:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:48.493459 24871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:48.493687 24871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:48.498801 24886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:48.499090 24871 server_base.cc:1034] running on GCE node
W20260809 23:16:48.499316 24881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:48.499406 24880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:48.499671 24871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:48.499923 24871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:48.501071 24871 hybrid_clock.cc:648] HybridClock initialized: now 1786317408501048 us; error 42 us; skew 500 ppm
Aug 09 23:16:48 dist-test-slave-7vjv krb5kdc[22662](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317408, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:48.503741 24871 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:16:48.504212 24871 webserver.cc:460] Webserver started at http://127.2.76.195:35173/ using document root <none> and password file <none>
I20260809 23:16:48.504420 24871 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:48.504492 24871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 23:16:48 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317408, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:48.524326 24871 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.001s
I20260809 23:16:48.530093 24899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:48.530295 24871 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20260809 23:16:48.530406 24871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "b8aea4dce8b7435786d9dbeefd11e032"
format_stamp: "Formatted at 2026-08-09 23:16:47 on dist-test-slave-7vjv"
server_key: "8310c3e7c64eab3120c6eb52813a5304"
server_key_iv: "0b989dd00958522f8e2677af71cb666b"
server_key_version: "kuduclusterkey@0"
I20260809 23:16:48.530889 24871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:48.547768 24871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:48.550484 24871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:48.550619 24871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:48.550818 24871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:48.551178 24871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:48.551231 24871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:48.551257 24871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:48.551328 24871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:48.558559 24871 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:34573
I20260809 23:16:48.559099 24871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:16:48.561120 25092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:34573 every 8 connection(s)
I20260809 23:16:48.564777 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 24871
Aug 09 23:16:48 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317408, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:48.570377 24660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:48.559826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:43129 (local address 127.2.76.254:41229)
0809 23:16:48.559868 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:48.559870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:48.560360 (+ 490us) server_negotiation.cc:407] Connection header received
0809 23:16:48.560969 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:48.560970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:48.560982 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:48.561046 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:48.561732 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:48.562431 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:48.563674 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:48.563962 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:48.563998 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:48.566841 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:48.566842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:48.566843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:48.566851 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:48.568792 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:48.569310 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:48.569311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:48.569312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:48.569374 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:48.569681 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:48.569682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:48.569682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:48.569823 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:48.569902 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:48.570220 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 23:16:48.570293 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:16:48.570513 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:48.559825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:41229 (local address 127.2.76.195:43129)
0809 23:16:48.560222 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:48.560266 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:48.560855 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:48.561166 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:48.561170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:48.561406 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:48.561630 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:48.561635 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:48.562519 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:48.562520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:48.563563 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:48.563565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:48.564112 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:48.564113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:48.564233 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:48.565032 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:48.565041 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:48.566720 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:48.568904 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:48.568910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:48.568912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:48.569200 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:48.569443 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:48.569444 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:48.569444 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:48.569600 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:48.569915 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:48.569921 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:48.570065 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 23:16:48.570184 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 23:16:48.570322 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":84,"threads_started":1}
I20260809 23:16:48.571058 25095 heartbeater.cc:347] Connected to a master server at 127.2.76.254:41229
I20260809 23:16:48.571144 25095 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:48.571300 25095 heartbeater.cc:511] Master 127.2.76.254:41229 requested a full tablet report, sending...
W20260809 23:16:48.571949 25095 heartbeater.cc:644] Failed to heartbeat to 127.2.76.254:41229 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 23:16:48.572028 25095 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:48.572088 25095 heartbeater.cc:511] Master 127.2.76.254:41229 requested a full tablet report, sending...
I20260809 23:16:48.572484 25095 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:48.572559 25095 heartbeater.cc:511] Master 127.2.76.254:41229 requested a full tablet report, sending...
W20260809 23:16:48.572928 25095 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 23:16:48.583227 24660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:48.565148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43306 (local address 127.2.76.254:41229)
0809 23:16:48.570443 (+ 5295us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:48.570445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:48.576302 (+ 5857us) server_negotiation.cc:407] Connection header received
0809 23:16:48.576385 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:48.576386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:48.576396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:48.576454 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:48.577161 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:48.577601 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:48.578629 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:48.578906 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:48.578980 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:48.580430 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:48.580432 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:48.580432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:48.580443 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:48.582077 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:48.582448 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:48.582449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:48.582450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:48.582508 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:48.582710 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:48.582711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:48.582711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:48.582842 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:48.583024 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:48.583058 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 23:16:48.583124 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5282,"thread_start_us":45,"threads_started":1}
Aug 09 23:16:49 dist-test-slave-7vjv krb5kdc[22662](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786317409, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:49.198056 24657 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:49.198201 24657 heartbeater.cc:511] Master 127.2.76.254:41229 requested a full tablet report, sending...
I20260809 23:16:49.198544 24330 ts_manager.cc:194] Registered new tserver with Master: c647355fe82f4cad9d83a7d8a3717420 (127.2.76.193:46055)
I20260809 23:16:49.199147 24330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:59157
I20260809 23:16:49.380272 24866 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:49.380422 24866 heartbeater.cc:511] Master 127.2.76.254:41229 requested a full tablet report, sending...
I20260809 23:16:49.380831 24330 ts_manager.cc:194] Registered new tserver with Master: b750e93d33e44b3892c6365fcbdee4b6 (127.2.76.194:43461)
I20260809 23:16:49.381354 24330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:43315
Aug 09 23:16:49 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317409, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
I20260809 23:16:49.573107 25095 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:49.573235 25095 heartbeater.cc:511] Master 127.2.76.254:41229 requested a full tablet report, sending...
I20260809 23:16:49.573566 24330 ts_manager.cc:194] Registered new tserver with Master: b8aea4dce8b7435786d9dbeefd11e032 (127.2.76.195:34573)
I20260809 23:16:49.574018 24330 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:43129
I20260809 23:16:49.579852 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:16:49.587807 25129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:49.580802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46658 (local address 127.2.76.254:41229)
0809 23:16:49.580914 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:49.580916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:49.580951 (+ 35us) server_negotiation.cc:407] Connection header received
0809 23:16:49.581009 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:49.581010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:49.581021 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:49.581089 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:49.581685 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.582356 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.583251 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.583530 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.583589 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:49.584928 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:49.584930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:49.584930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:49.584937 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:49.586561 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:49.586958 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:49.586959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:49.586960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:49.587023 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:49.587241 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:49.587242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:49.587242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:49.587368 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:49.587458 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:49.587603 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 23:16:49.587686 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":68,"threads_started":1}
I20260809 23:16:49.589730 24330 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:16:49.591755 24330 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:16:49.604645 25141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:49.594608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:43461 (local address 127.0.0.1:44508)
0809 23:16:49.599992 (+ 5384us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:49.600000 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:49.600078 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:49.600431 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:49.600434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:49.600436 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:49.600493 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:49.600495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.601924 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.601925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:49.603037 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:49.603039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.604402 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.604403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:49.604472 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:49.604482 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 23:16:49.604538 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 23:16:49.604578 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5372,"thread_start_us":73,"threads_started":1}
I20260809 23:16:49.605105 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:49.594704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44508 (local address 127.2.76.194:43461)
0809 23:16:49.599947 (+ 5243us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:49.599951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:49.600068 (+ 117us) server_negotiation.cc:407] Connection header received
0809 23:16:49.600193 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:49.600196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:49.600231 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:49.600312 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:49.600571 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.601842 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.603347 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.604263 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.604320 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:49.604372 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:49.604990 (+ 618us) server_negotiation.cc:299] Negotiation successful
0809 23:16:49.605036 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5211,"thread_start_us":52,"threads_started":1}
I20260809 23:16:49.605839 24750 tablet_service.cc:1467] Processing CreateTablet for tablet f62dc0203cc2486da44d5d0f174c2889 (DEFAULT_TABLE table=test-table [id=b89dec1b6798440bae83432fcf90fd26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:16:49.606158 25143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:49.594570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:46055 (local address 127.0.0.1:52064)
0809 23:16:49.595023 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:49.595060 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:49.595118 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:49.600990 (+ 5872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:49.600990 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:49.600992 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:49.601018 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:49.601019 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.602022 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.602023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:49.604963 (+ 2940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:49.604966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.605712 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.605713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:49.605778 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:49.605782 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:16:49.606061 (+ 279us) client_negotiation.cc:240] Negotiation successful
0809 23:16:49.606095 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":2018,"thread_start_us":71,"threads_started":1}
I20260809 23:16:49.606212 24750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f62dc0203cc2486da44d5d0f174c2889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:49.606388 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:49.594798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52064 (local address 127.2.76.193:46055)
0809 23:16:49.600406 (+ 5608us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:49.600411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:49.600421 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:16:49.600451 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:49.600453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:49.600465 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:49.600541 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:49.601091 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.601950 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.605039 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.605635 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.605712 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:49.605758 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:49.606285 (+ 527us) server_negotiation.cc:299] Negotiation successful
0809 23:16:49.606320 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5569,"thread_start_us":55,"threads_started":1}
I20260809 23:16:49.606787 25139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:49.594780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:34573 (local address 127.0.0.1:59764)
0809 23:16:49.600209 (+ 5429us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:49.600234 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:49.600273 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:49.600879 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:49.600880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:49.600882 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:49.600927 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:49.600929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.603097 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.603098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:49.604017 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:49.604020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.606608 (+ 2588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.606609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:49.606659 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:49.606663 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:16:49.606704 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 23:16:49.606734 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5415,"thread_start_us":72,"threads_started":1}
I20260809 23:16:49.607026 25152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:49.594960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59764 (local address 127.2.76.195:34573)
0809 23:16:49.600660 (+ 5700us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:49.600664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:49.600672 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:16:49.600700 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:49.600701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:49.600715 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:49.600794 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:49.602103 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.603010 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.605790 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.606464 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.606532 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:49.606577 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:49.606923 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 23:16:49.606961 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5671,"thread_start_us":40,"threads_started":1}
I20260809 23:16:49.606964 24544 tablet_service.cc:1467] Processing CreateTablet for tablet f62dc0203cc2486da44d5d0f174c2889 (DEFAULT_TABLE table=test-table [id=b89dec1b6798440bae83432fcf90fd26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:16:49.607280 24544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f62dc0203cc2486da44d5d0f174c2889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:49.607540 24984 tablet_service.cc:1467] Processing CreateTablet for tablet f62dc0203cc2486da44d5d0f174c2889 (DEFAULT_TABLE table=test-table [id=b89dec1b6798440bae83432fcf90fd26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:16:49.607805 24984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f62dc0203cc2486da44d5d0f174c2889. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:49.610878 25156 tablet_bootstrap.cc:492] T f62dc0203cc2486da44d5d0f174c2889 P c647355fe82f4cad9d83a7d8a3717420: Bootstrap starting.
I20260809 23:16:49.611373 25156 tablet_bootstrap.cc:654] T f62dc0203cc2486da44d5d0f174c2889 P c647355fe82f4cad9d83a7d8a3717420: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:49.611657 25156 log.cc:824] T f62dc0203cc2486da44d5d0f174c2889 P c647355fe82f4cad9d83a7d8a3717420: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:49.611960 25161 tablet_bootstrap.cc:492] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032: Bootstrap starting.
I20260809 23:16:49.612205 25154 tablet_bootstrap.cc:492] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6: Bootstrap starting.
I20260809 23:16:49.612665 25161 tablet_bootstrap.cc:654] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:49.612747 25154 tablet_bootstrap.cc:654] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:49.613020 25161 log.cc:824] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:49.613057 25154 log.cc:824] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:49.613510 25156 tablet_bootstrap.cc:492] T f62dc0203cc2486da44d5d0f174c2889 P c647355fe82f4cad9d83a7d8a3717420: No bootstrap required, opened a new log
I20260809 23:16:49.613586 25156 ts_tablet_manager.cc:1397] T f62dc0203cc2486da44d5d0f174c2889 P c647355fe82f4cad9d83a7d8a3717420: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 23:16:49.615450 25156 raft_consensus.cc:348] T f62dc0203cc2486da44d5d0f174c2889 P c647355fe82f4cad9d83a7d8a3717420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c647355fe82f4cad9d83a7d8a3717420" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46055 } } peers { permanent_uuid: "b8aea4dce8b7435786d9dbeefd11e032" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 34573 } } peers { permanent_uuid: "b750e93d33e44b3892c6365fcbdee4b6" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43461 } }
I20260809 23:16:49.615581 25156 raft_consensus.cc:374] T f62dc0203cc2486da44d5d0f174c2889 P c647355fe82f4cad9d83a7d8a3717420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:49.615635 25156 raft_consensus.cc:729] T f62dc0203cc2486da44d5d0f174c2889 P c647355fe82f4cad9d83a7d8a3717420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c647355fe82f4cad9d83a7d8a3717420, State: Initialized, Role: FOLLOWER
I20260809 23:16:49.615762 25156 consensus_queue.cc:260] T f62dc0203cc2486da44d5d0f174c2889 P c647355fe82f4cad9d83a7d8a3717420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c647355fe82f4cad9d83a7d8a3717420" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46055 } } peers { permanent_uuid: "b8aea4dce8b7435786d9dbeefd11e032" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 34573 } } peers { permanent_uuid: "b750e93d33e44b3892c6365fcbdee4b6" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43461 } }
I20260809 23:16:49.616029 24657 heartbeater.cc:503] Master 127.2.76.254:41229 was elected leader, sending a full tablet report...
I20260809 23:16:49.616021 25156 ts_tablet_manager.cc:1428] T f62dc0203cc2486da44d5d0f174c2889 P c647355fe82f4cad9d83a7d8a3717420: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:16:49.617069 25161 tablet_bootstrap.cc:492] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032: No bootstrap required, opened a new log
I20260809 23:16:49.617197 25161 ts_tablet_manager.cc:1397] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260809 23:16:49.618454 25161 raft_consensus.cc:348] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c647355fe82f4cad9d83a7d8a3717420" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46055 } } peers { permanent_uuid: "b8aea4dce8b7435786d9dbeefd11e032" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 34573 } } peers { permanent_uuid: "b750e93d33e44b3892c6365fcbdee4b6" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43461 } }
I20260809 23:16:49.618573 25161 raft_consensus.cc:374] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:49.618639 25161 raft_consensus.cc:729] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8aea4dce8b7435786d9dbeefd11e032, State: Initialized, Role: FOLLOWER
I20260809 23:16:49.618745 25161 consensus_queue.cc:260] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c647355fe82f4cad9d83a7d8a3717420" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46055 } } peers { permanent_uuid: "b8aea4dce8b7435786d9dbeefd11e032" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 34573 } } peers { permanent_uuid: "b750e93d33e44b3892c6365fcbdee4b6" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43461 } }
I20260809 23:16:49.618978 25095 heartbeater.cc:503] Master 127.2.76.254:41229 was elected leader, sending a full tablet report...
I20260809 23:16:49.618968 25161 ts_tablet_manager.cc:1428] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 23:16:49.619946 25154 tablet_bootstrap.cc:492] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6: No bootstrap required, opened a new log
I20260809 23:16:49.620009 25154 ts_tablet_manager.cc:1397] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260809 23:16:49.621605 25154 raft_consensus.cc:348] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c647355fe82f4cad9d83a7d8a3717420" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46055 } } peers { permanent_uuid: "b8aea4dce8b7435786d9dbeefd11e032" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 34573 } } peers { permanent_uuid: "b750e93d33e44b3892c6365fcbdee4b6" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43461 } }
I20260809 23:16:49.621694 25154 raft_consensus.cc:374] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:49.621711 25154 raft_consensus.cc:729] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b750e93d33e44b3892c6365fcbdee4b6, State: Initialized, Role: FOLLOWER
I20260809 23:16:49.621802 25154 consensus_queue.cc:260] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c647355fe82f4cad9d83a7d8a3717420" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46055 } } peers { permanent_uuid: "b8aea4dce8b7435786d9dbeefd11e032" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 34573 } } peers { permanent_uuid: "b750e93d33e44b3892c6365fcbdee4b6" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43461 } }
I20260809 23:16:49.622015 25154 ts_tablet_manager.cc:1428] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 23:16:49.622026 24866 heartbeater.cc:503] Master 127.2.76.254:41229 was elected leader, sending a full tablet report...
Aug 09 23:16:49 dist-test-slave-7vjv krb5kdc[22662](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317409, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for HTTP/127.2.76.212@KRBTEST.COM
W20260809 23:16:49.683810 24658 tablet.cc:2369] T f62dc0203cc2486da44d5d0f174c2889 P c647355fe82f4cad9d83a7d8a3717420: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:16:49.809589 25169 raft_consensus.cc:482] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:16:49.809757 25169 raft_consensus.cc:504] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c647355fe82f4cad9d83a7d8a3717420" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46055 } } peers { permanent_uuid: "b8aea4dce8b7435786d9dbeefd11e032" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 34573 } } peers { permanent_uuid: "b750e93d33e44b3892c6365fcbdee4b6" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43461 } }
I20260809 23:16:49.810182 25169 leader_election.cc:288] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c647355fe82f4cad9d83a7d8a3717420 (127.2.76.193:46055), b750e93d33e44b3892c6365fcbdee4b6 (127.2.76.194:43461)
W20260809 23:16:49.812145 25096 tablet.cc:2369] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:16:49.814150 25147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:49.810574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:49885 (local address 127.2.76.193:46055)
0809 23:16:49.810617 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:49.810619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:49.811095 (+ 476us) server_negotiation.cc:407] Connection header received
0809 23:16:49.811183 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:49.811184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:49.811195 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:49.811221 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:49.811427 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.812093 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.813334 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.813804 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.813847 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:49.813856 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:49.814051 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 23:16:49.814089 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:16:49.814539 25177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:49.810366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:46055 (local address 127.2.76.195:49885)
0809 23:16:49.811036 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:49.811046 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:49.811120 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:49.811280 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:49.811280 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:49.811282 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:49.811364 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:49.811365 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.812202 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.812203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:49.813244 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:49.813247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.813925 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.813926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:49.813984 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:49.813987 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:16:49.814387 (+ 400us) client_negotiation.cc:240] Negotiation successful
0809 23:16:49.814459 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":104,"threads_started":1}
I20260809 23:16:49.814950 24571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62dc0203cc2486da44d5d0f174c2889" candidate_uuid: "b8aea4dce8b7435786d9dbeefd11e032" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c647355fe82f4cad9d83a7d8a3717420" is_pre_election: true
I20260809 23:16:49.815112 24571 raft_consensus.cc:2454] T f62dc0203cc2486da44d5d0f174c2889 P c647355fe82f4cad9d83a7d8a3717420 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8aea4dce8b7435786d9dbeefd11e032 in term 0.
I20260809 23:16:49.815441 24904 leader_election.cc:302] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8aea4dce8b7435786d9dbeefd11e032, c647355fe82f4cad9d83a7d8a3717420; no voters:
I20260809 23:16:49.815553 25169 raft_consensus.cc:2781] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:16:49.815589 25169 raft_consensus.cc:482] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:16:49.815608 25169 raft_consensus.cc:3037] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:49.816421 25142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:49.810967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:52935 (local address 127.2.76.194:43461)
0809 23:16:49.811026 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:49.811028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:49.813465 (+ 2437us) server_negotiation.cc:407] Connection header received
0809 23:16:49.813568 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:49.813569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:49.813581 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:49.813606 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:49.813791 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.814361 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.815533 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.816030 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.816077 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:49.816085 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:49.816318 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 23:16:49.816359 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:16:49.816521 25169 raft_consensus.cc:504] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c647355fe82f4cad9d83a7d8a3717420" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46055 } } peers { permanent_uuid: "b8aea4dce8b7435786d9dbeefd11e032" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 34573 } } peers { permanent_uuid: "b750e93d33e44b3892c6365fcbdee4b6" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43461 } }
I20260809 23:16:49.816638 25169 leader_election.cc:288] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [CANDIDATE]: Term 1 election: Requested vote from peers c647355fe82f4cad9d83a7d8a3717420 (127.2.76.193:46055), b750e93d33e44b3892c6365fcbdee4b6 (127.2.76.194:43461)
I20260809 23:16:49.816648 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:49.810799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:43461 (local address 127.2.76.195:52935)
0809 23:16:49.813393 (+ 2594us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:49.813403 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:49.813489 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:49.813663 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:49.813664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:49.813665 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:49.813721 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:49.813723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.814459 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.814460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:49.815425 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:49.815428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.816174 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.816174 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:49.816236 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:49.816239 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:16:49.816537 (+ 298us) client_negotiation.cc:240] Negotiation successful
0809 23:16:49.816575 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2575,"thread_start_us":76,"threads_started":1}
I20260809 23:16:49.817035 24789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62dc0203cc2486da44d5d0f174c2889" candidate_uuid: "b8aea4dce8b7435786d9dbeefd11e032" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b750e93d33e44b3892c6365fcbdee4b6" is_pre_election: true
I20260809 23:16:49.817167 24789 raft_consensus.cc:2454] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8aea4dce8b7435786d9dbeefd11e032 in term 0.
I20260809 23:16:49.817216 24571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62dc0203cc2486da44d5d0f174c2889" candidate_uuid: "b8aea4dce8b7435786d9dbeefd11e032" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c647355fe82f4cad9d83a7d8a3717420"
I20260809 23:16:49.817256 24786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f62dc0203cc2486da44d5d0f174c2889" candidate_uuid: "b8aea4dce8b7435786d9dbeefd11e032" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b750e93d33e44b3892c6365fcbdee4b6"
I20260809 23:16:49.817299 24786 raft_consensus.cc:3037] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:49.817284 24571 raft_consensus.cc:3037] T f62dc0203cc2486da44d5d0f174c2889 P c647355fe82f4cad9d83a7d8a3717420 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:49.818066 24786 raft_consensus.cc:2454] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8aea4dce8b7435786d9dbeefd11e032 in term 1.
I20260809 23:16:49.818073 24571 raft_consensus.cc:2454] T f62dc0203cc2486da44d5d0f174c2889 P c647355fe82f4cad9d83a7d8a3717420 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8aea4dce8b7435786d9dbeefd11e032 in term 1.
I20260809 23:16:49.818219 24904 leader_election.cc:302] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8aea4dce8b7435786d9dbeefd11e032, c647355fe82f4cad9d83a7d8a3717420; no voters:
I20260809 23:16:49.818284 25169 raft_consensus.cc:2781] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:16:49.818405 25169 raft_consensus.cc:686] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [term 1 LEADER]: Becoming Leader. State: Replica: b8aea4dce8b7435786d9dbeefd11e032, State: Running, Role: LEADER
I20260809 23:16:49.818490 25169 consensus_queue.cc:237] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c647355fe82f4cad9d83a7d8a3717420" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46055 } } peers { permanent_uuid: "b8aea4dce8b7435786d9dbeefd11e032" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 34573 } } peers { permanent_uuid: "b750e93d33e44b3892c6365fcbdee4b6" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43461 } }
I20260809 23:16:49.819293 24330 catalog_manager.cc:5495] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8aea4dce8b7435786d9dbeefd11e032 (127.2.76.195). New cstate: current_term: 1 leader_uuid: "b8aea4dce8b7435786d9dbeefd11e032" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c647355fe82f4cad9d83a7d8a3717420" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8aea4dce8b7435786d9dbeefd11e032" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 34573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b750e93d33e44b3892c6365fcbdee4b6" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43461 } health_report { overall_health: UNKNOWN } } }
I20260809 23:16:49.830753 25152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:49.828014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59770 (local address 127.2.76.195:34573)
0809 23:16:49.828056 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:49.828058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:49.828066 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:16:49.828123 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:49.828124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:49.828141 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:49.828211 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:16:49.828351 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.828922 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.829928 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:49.830169 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:49.830226 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:49.830422 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:16:49.830536 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:16:49.830654 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:49.830676 (+ 22us) server_negotiation.cc:299] Negotiation successful
0809 23:16:49.830696 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:16:49.832520 24571 raft_consensus.cc:1261] T f62dc0203cc2486da44d5d0f174c2889 P c647355fe82f4cad9d83a7d8a3717420 [term 1 FOLLOWER]: Refusing update from remote peer b8aea4dce8b7435786d9dbeefd11e032: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:16:49.832543 24786 raft_consensus.cc:1261] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6 [term 1 FOLLOWER]: Refusing update from remote peer b8aea4dce8b7435786d9dbeefd11e032: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:16:49.832824 25179 consensus_queue.cc:1035] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c647355fe82f4cad9d83a7d8a3717420" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:16:49.832983 25169 consensus_queue.cc:1035] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b750e93d33e44b3892c6365fcbdee4b6" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:16:49.837278 25187 mvcc.cc:205] Tried to move back new op lower bound from 7316756110671425536 to 7316756110617059328. Current Snapshot: MvccSnapshot[applied={T|T < 7316756110671425536}]
I20260809 23:16:49.838543 25194 mvcc.cc:205] Tried to move back new op lower bound from 7316756110671425536 to 7316756110617059328. Current Snapshot: MvccSnapshot[applied={T|T < 7316756110671425536}]
I20260809 23:16:49.842289 24330 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:16:49.842378 24330 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:16:49.843152 24330 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d717204bbbbf435f83c832078bcbcd6f: Sending DeleteTablet for 3 replicas of tablet f62dc0203cc2486da44d5d0f174c2889
I20260809 23:16:49.843501 24750 tablet_service.cc:1514] Processing DeleteTablet for tablet f62dc0203cc2486da44d5d0f174c2889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:16:49 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:44508
I20260809 23:16:49.843721 24984 tablet_service.cc:1514] Processing DeleteTablet for tablet f62dc0203cc2486da44d5d0f174c2889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:16:49 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:59764
I20260809 23:16:49.844247 24544 tablet_service.cc:1514] Processing DeleteTablet for tablet f62dc0203cc2486da44d5d0f174c2889 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:16:49 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:52064
I20260809 23:16:49.844394 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 24385
I20260809 23:16:49.855647 25206 tablet_replica.cc:330] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032: stopping tablet replica
I20260809 23:16:49.855753 25206 raft_consensus.cc:2229] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:16:49.855860 25206 raft_consensus.cc:2258] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:16:49.856843 25206 ts_tablet_manager.cc:1905] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:16:49.858803 25206 ts_tablet_manager.cc:1918] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:16:49.858927 25206 log.cc:1196] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/f62dc0203cc2486da44d5d0f174c2889
I20260809 23:16:49.859290 25206 ts_tablet_manager.cc:1939] T f62dc0203cc2486da44d5d0f174c2889 P b8aea4dce8b7435786d9dbeefd11e032: Deleting consensus metadata
I20260809 23:16:49.859733 24304 catalog_manager.cc:4841] TS b8aea4dce8b7435786d9dbeefd11e032 (127.2.76.195:34573): tablet f62dc0203cc2486da44d5d0f174c2889 (table test-table [id=b89dec1b6798440bae83432fcf90fd26]) successfully deleted
I20260809 23:16:49.859951 25203 tablet_replica.cc:330] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6: stopping tablet replica
I20260809 23:16:49.864192 25203 raft_consensus.cc:2229] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:16:49.864372 25203 raft_consensus.cc:2258] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:16:49.864828 25203 ts_tablet_manager.cc:1905] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:16:49.866379 25203 ts_tablet_manager.cc:1918] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:16:49.866451 25203 log.cc:1196] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/f62dc0203cc2486da44d5d0f174c2889
I20260809 23:16:49.866711 25203 ts_tablet_manager.cc:1939] T f62dc0203cc2486da44d5d0f174c2889 P b750e93d33e44b3892c6365fcbdee4b6: Deleting consensus metadata
I20260809 23:16:49.867115 24312 catalog_manager.cc:4841] TS b750e93d33e44b3892c6365fcbdee4b6 (127.2.76.194:43461): tablet f62dc0203cc2486da44d5d0f174c2889 (table test-table [id=b89dec1b6798440bae83432fcf90fd26]) successfully deleted
W20260809 23:16:49.867585 24308 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:46055 (error 108)
I20260809 23:16:49.867910 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 24661
I20260809 23:16:49.876839 25139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:49.876644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:46055 (local address 127.0.0.1:52070)
0809 23:16:49.876762 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:49.876803 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:46055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260809 23:16:49.877032 24308 catalog_manager.cc:4584] TS c647355fe82f4cad9d83a7d8a3717420 (127.2.76.193:46055): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f62dc0203cc2486da44d5d0f174c2889: Network error: Client connection negotiation failed: client connection to 127.2.76.193:46055: connect: Connection refused (error 111)
I20260809 23:16:49.877928 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 24871
I20260809 23:16:49.887064 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 24279
I20260809 23:16:49.897094 2355 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 23:16:50.246008 2355 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 23:16:50.246134 2355 mini_ranger.cc:66] Stopping Ranger...
I20260809 23:16:50.591002 2355 mini_ranger.cc:68] Stopped Ranger
2026-08-09 23:16:50.591 UTC [22718] LOG: received smart shutdown request
2026-08-09 23:16:50.595 UTC [22718] LOG: background worker "logical replication launcher" (PID 22733) exited with exit code 1
2026-08-09 23:16:50.596 UTC [22724] LOG: shutting down
2026-08-09 23:16:50.733 UTC [22718] LOG: database system is shut down
2026-08-09T23:16:50Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (44303 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:16:50 dist-test-slave-7vjv krb5kdc[25237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:16:50 dist-test-slave-7vjv krb5kdc[25237](info): set up 2 sockets
Aug 09 23:16:50 dist-test-slave-7vjv krb5kdc[25237](info): commencing operation
krb5kdc: starting...
W20260809 23:16:51.973547 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:16:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:16:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:16:52.078097 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:46517
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:46517,127.2.76.253:45317,127.2.76.252:43173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:52.147744 25256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:52.147908 25256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:52.147927 25256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:52.148945 25256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:16:52.148980 25256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:52.148993 25256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:52.149003 25256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:16:52.149045 25256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:16:52.150372 25256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:46517,127.2.76.253:45317,127.2.76.252:43173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:46517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:52.150611 25256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:52.150801 25256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:52.154986 25263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:52.154983 25265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:52.155058 25262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:52.155170 25256 server_base.cc:1034] running on GCE node
I20260809 23:16:52.155443 25256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:52.155645 25256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:52.156766 25256 hybrid_clock.cc:648] HybridClock initialized: now 1786317412156747 us; error 35 us; skew 500 ppm
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:52.158833 25256 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:16:52.159214 25256 webserver.cc:460] Webserver started at http://127.2.76.254:45155/ using document root <none> and password file <none>
I20260809 23:16:52.159406 25256 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:52.159466 25256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:52.159611 25256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:16:52.160609 25256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01"
format_stamp: "Formatted at 2026-08-09 23:16:52 on dist-test-slave-7vjv"
server_key: "c383caf5e00a51a67d62c5dcc38da7d0"
server_key_iv: "ab3b8584fe19faa837b4f87c7fe9bb7a"
server_key_version: "encryptionkey@0"
I20260809 23:16:52.160914 25256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01"
format_stamp: "Formatted at 2026-08-09 23:16:52 on dist-test-slave-7vjv"
server_key: "c383caf5e00a51a67d62c5dcc38da7d0"
server_key_iv: "ab3b8584fe19faa837b4f87c7fe9bb7a"
server_key_version: "encryptionkey@0"
I20260809 23:16:52.162081 25256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:16:52.162681 25280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:52.162840 25256 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:16:52.162921 25256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01"
format_stamp: "Formatted at 2026-08-09 23:16:52 on dist-test-slave-7vjv"
server_key: "c383caf5e00a51a67d62c5dcc38da7d0"
server_key_iv: "ab3b8584fe19faa837b4f87c7fe9bb7a"
server_key_version: "encryptionkey@0"
I20260809 23:16:52.162981 25256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:52.175985 25256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:52.177695 25256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:52.177846 25256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:52.181454 25256 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:46517
I20260809 23:16:52.181510 25373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:46517 every 8 connection(s)
I20260809 23:16:52.181805 25256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:16:52.182621 25374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:52.183805 25374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:52.191555 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.184365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:46517 (local address 127.0.0.1:43536)
0809 23:16:52.184518 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.184555 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:52.184919 (+ 364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:52.185127 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:52.185129 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:52.185319 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:52.185453 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.185457 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.186355 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.186356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.186985 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.186986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.187397 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.187398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.187445 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.187877 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:52.187884 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:52.189112 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:52.190439 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.190445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.190447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.190620 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.190811 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.190812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.190812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.190885 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.191146 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:52.191150 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:52.191196 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 23:16:52.191258 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 23:16:52.191396 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":1,"thread_start_us":64,"threads_started":1}
I20260809 23:16:52.191560 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.184406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43536 (local address 127.2.76.254:46517)
0809 23:16:52.184587 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:52.184591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:52.184617 (+ 26us) server_negotiation.cc:407] Connection header received
0809 23:16:52.185021 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:52.185022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:52.185036 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:52.185101 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:52.185526 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.186279 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.187051 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.187336 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.187398 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.189181 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:52.189188 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:52.189188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:52.189193 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:52.190372 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.190685 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.190687 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.190687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.190758 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.190933 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.190933 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.190933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.191096 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:52.191150 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:52.191291 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 23:16:52.191443 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":156,"thread_start_us":103,"threads_started":1}
I20260809 23:16:52.191795 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 25256
I20260809 23:16:52.191877 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:16:52.192016 2355 external_mini_cluster.cc:1428] Setting key e9a9e0dfca207b8c5748eff6e9a78dfa
I20260809 23:16:52.192746 25374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:16:52.193019 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.192925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:45317 (local address 127.0.0.1:58756)
0809 23:16:52.192961 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.192999 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 23:16:52.193146 25283 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.76.253:45317: connect: Connection refused (error 111)
I20260809 23:16:52.193516 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.193443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:45317 (local address 127.0.0.1:58760)
0809 23:16:52.193480 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.193501 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 23:16:52.193614 25374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.253:45317: Network error: Client connection negotiation failed: client connection to 127.2.76.253:45317: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.2.76.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.253.keytab.
Entry for principal kudu/127.2.76.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.253.keytab.
I20260809 23:16:52.214960 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.253:45317
--webserver_interface=127.2.76.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.253.keytab
--principal=kudu/127.2.76.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:46517,127.2.76.253:45317,127.2.76.252:43173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:16:52.242817 25374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } attempt: 1
I20260809 23:16:52.243237 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.243115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:45317 (local address 127.0.0.1:58770)
0809 23:16:52.243171 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.243208 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 23:16:52.243572 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.243481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:45317 (local address 127.0.0.1:58776)
0809 23:16:52.243526 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.243550 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 23:16:52.243681 25374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.253:45317: Network error: Client connection negotiation failed: client connection to 127.2.76.253:45317: connect: Connection refused (error 111)
W20260809 23:16:52.289533 25388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:52.289728 25388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:52.289765 25388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:52.290742 25388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:16:52.290812 25388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:52.290839 25388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:52.290866 25388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:16:52.290889 25388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:16:52.292168 25388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:46517,127.2.76.253:45317,127.2.76.252:43173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.253.keytab
--principal=kudu/127.2.76.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.253:45317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.76.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:52.292400 25388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:52.292606 25388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:52.296757 25398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:52.296769 25395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:52.296854 25388 server_base.cc:1034] running on GCE node
W20260809 23:16:52.296775 25394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:52.297227 25388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:52.297482 25388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:52.298677 25388 hybrid_clock.cc:648] HybridClock initialized: now 1786317412298666 us; error 35 us; skew 500 ppm
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:52.300912 25388 init.cc:377] Logged in from keytab as kudu/127.2.76.253@KRBTEST.COM (short username kudu)
I20260809 23:16:52.301301 25388 webserver.cc:460] Webserver started at http://127.2.76.253:40005/ using document root <none> and password file <none>
I20260809 23:16:52.301492 25388 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:52.301546 25388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:52.301671 25388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:16:52.302510 25388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/data/instance:
uuid: "fc6d1acc11454d96b19a84d51199dbc1"
format_stamp: "Formatted at 2026-08-09 23:16:52 on dist-test-slave-7vjv"
server_key: "f5fcb18cce793bbe3cc8a4f3b3e9a8f7"
server_key_iv: "8c8c7339f0ae31fcd8bd9adf00303f7f"
server_key_version: "encryptionkey@0"
I20260809 23:16:52.302814 25388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/wal/instance:
uuid: "fc6d1acc11454d96b19a84d51199dbc1"
format_stamp: "Formatted at 2026-08-09 23:16:52 on dist-test-slave-7vjv"
server_key: "f5fcb18cce793bbe3cc8a4f3b3e9a8f7"
server_key_iv: "8c8c7339f0ae31fcd8bd9adf00303f7f"
server_key_version: "encryptionkey@0"
I20260809 23:16:52.303925 25388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:16:52.304600 25406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:52.304746 25388 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:52.304836 25388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/wal
uuid: "fc6d1acc11454d96b19a84d51199dbc1"
format_stamp: "Formatted at 2026-08-09 23:16:52 on dist-test-slave-7vjv"
server_key: "f5fcb18cce793bbe3cc8a4f3b3e9a8f7"
server_key_iv: "8c8c7339f0ae31fcd8bd9adf00303f7f"
server_key_version: "encryptionkey@0"
I20260809 23:16:52.304903 25388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:52.311820 25374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } attempt: 2
I20260809 23:16:52.312160 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.312029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:45317 (local address 127.0.0.1:58792)
0809 23:16:52.312073 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.312131 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 23:16:52.312417 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.312341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:45317 (local address 127.0.0.1:58804)
0809 23:16:52.312377 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.312398 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 23:16:52.312494 25374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.253:45317: Network error: Client connection negotiation failed: client connection to 127.2.76.253:45317: connect: Connection refused (error 111)
I20260809 23:16:52.319372 25388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:52.321353 25388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:52.321487 25388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:52.325034 25388 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.253:45317
I20260809 23:16:52.325066 25490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.253:45317 every 8 connection(s)
I20260809 23:16:52.325349 25388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/data/info.pb
I20260809 23:16:52.326042 25491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:52.327260 25491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:16:52.329489 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 25388
I20260809 23:16:52.329600 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-1/wal/instance
I20260809 23:16:52.329775 2355 external_mini_cluster.cc:1428] Setting key dfd69ba6e453119416e28ed999c382dd
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.253@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:52.335800 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.327812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43540 (local address 127.2.76.254:46517)
0809 23:16:52.327859 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:52.327861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:52.328036 (+ 175us) server_negotiation.cc:407] Connection header received
0809 23:16:52.328463 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:52.328463 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:52.328473 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:52.328510 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:52.328950 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.329354 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.330400 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.330599 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.330668 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.332782 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:52.332783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:52.332784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:52.332790 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:52.334309 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.334649 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.334650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.334651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.334708 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.334937 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.334937 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.334938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.335075 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:52.335142 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:52.335509 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 23:16:52.335647 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 23:16:52.335836 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.327790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:46517 (local address 127.0.0.1:43540)
0809 23:16:52.327932 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.327964 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:52.328378 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:52.328552 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:52.328554 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:52.328742 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:52.328891 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.328894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.329465 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.329466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.330300 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.330303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.330677 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.330678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.330793 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.331243 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:52.331253 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:52.332696 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:52.334387 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.334393 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.334395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.334581 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.334777 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.334778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.334778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.334862 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.335181 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:52.335189 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:52.335354 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 23:16:52.335460 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 23:16:52.335628 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260809 23:16:52.338327 25491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.76.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.252@KRBTEST.COM" created.
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.253@KRBTEST.COM for kudu/127.2.76.253@KRBTEST.COM
I20260809 23:16:52.345880 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.338579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:45317 (local address 127.0.0.1:58810)
0809 23:16:52.338721 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.338729 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:52.338785 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:52.338903 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:52.338903 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:52.339025 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:52.339060 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.339062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.340278 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.340279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.340798 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.340800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.341474 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.341475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.341537 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.341930 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:52.341934 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:52.343310 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:52.344864 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.344865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.344866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.345029 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.345223 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.345224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.345224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.345277 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.345583 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:52.345584 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:52.345619 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 23:16:52.345679 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 23:16:52.345766 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260809 23:16:52.345949 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.338610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.2.76.253:45317)
0809 23:16:52.338749 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:52.338768 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:52.338786 (+ 18us) server_negotiation.cc:407] Connection header received
0809 23:16:52.338835 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:52.338837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:52.338848 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:52.338904 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:52.339123 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.340189 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.340888 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.341398 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.341473 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.343376 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:52.343384 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:52.343384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:52.343390 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:52.344782 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.345105 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.345108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.345108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.345162 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.345341 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.345342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.345343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.345520 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:52.345603 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:52.345703 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 23:16:52.345814 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260809 23:16:52.346475 25491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:16:52.346753 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.346656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:43173 (local address 127.0.0.1:58786)
0809 23:16:52.346692 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.346729 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 23:16:52.346865 25411 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
I20260809 23:16:52.347226 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.347147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:43173 (local address 127.0.0.1:58798)
0809 23:16:52.347187 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.347207 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 23:16:52.347316 25491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.252:43173: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.252.keytab.
Entry for principal kudu/127.2.76.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.252.keytab.
I20260809 23:16:52.355978 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.252:43173
--webserver_interface=127.2.76.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.252.keytab
--principal=kudu/127.2.76.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:46517,127.2.76.253:45317,127.2.76.252:43173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:16:52.396481 25491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } attempt: 1
I20260809 23:16:52.396822 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.396730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:43173 (local address 127.0.0.1:58810)
0809 23:16:52.396773 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.396801 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 23:16:52.397116 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.397029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:43173 (local address 127.0.0.1:58824)
0809 23:16:52.397065 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.397098 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 23:16:52.397185 25491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.252:43173: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
I20260809 23:16:52.403605 25374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } attempt: 3
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for kudu/127.2.76.253@KRBTEST.COM
I20260809 23:16:52.409665 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.403862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:45317 (local address 127.0.0.1:58814)
0809 23:16:52.403899 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.403909 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:52.403965 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:52.404134 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:52.404135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:52.404271 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:52.404327 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.404329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.404942 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.404943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.405375 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.405376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.405698 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.405698 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.405770 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.406164 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:52.406168 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:52.407303 (+ 1135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:52.408707 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.408707 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.408708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.408864 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.409035 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.409036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.409037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.409090 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.409362 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:52.409362 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:52.409407 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 23:16:52.409482 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 23:16:52.409576 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 23:16:52.409708 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.403918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58814 (local address 127.2.76.253:45317)
0809 23:16:52.403970 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:52.403972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:52.404004 (+ 32us) server_negotiation.cc:407] Connection header received
0809 23:16:52.404025 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:52.404026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:52.404037 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:52.404086 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:52.404408 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.404857 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.405454 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.405635 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.405698 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.407366 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:52.407367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:52.407367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:52.407374 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:52.408630 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.408927 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.408928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.408928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.408978 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.409155 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.409156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.409156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.409276 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:52.409341 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:52.409546 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 23:16:52.409629 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 23:16:52.410043 25374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:16:52.410332 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.410247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:43173 (local address 127.0.0.1:58828)
0809 23:16:52.410289 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.410312 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 23:16:52.410636 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.410564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:43173 (local address 127.0.0.1:58842)
0809 23:16:52.410599 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.410619 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 23:16:52.410712 25374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.252:43173: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
W20260809 23:16:52.429373 25499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:52.429540 25499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:52.429584 25499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:52.430583 25499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:16:52.430619 25499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:52.430630 25499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:52.430670 25499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:16:52.430688 25499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:16:52.431938 25499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:46517,127.2.76.253:45317,127.2.76.252:43173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.252.keytab
--principal=kudu/127.2.76.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.252:43173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.76.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:52.432152 25499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:52.432358 25499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:52.436564 25508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:52.436563 25516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:52.436599 25512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:52.436810 25499 server_base.cc:1034] running on GCE node
I20260809 23:16:52.437042 25499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:52.437290 25499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:52.438441 25499 hybrid_clock.cc:648] HybridClock initialized: now 1786317412438431 us; error 23 us; skew 500 ppm
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:52.440661 25499 init.cc:377] Logged in from keytab as kudu/127.2.76.252@KRBTEST.COM (short username kudu)
I20260809 23:16:52.440995 25499 webserver.cc:460] Webserver started at http://127.2.76.252:37611/ using document root <none> and password file <none>
I20260809 23:16:52.441170 25499 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:52.441206 25499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:52.441284 25499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:16:52.441833 25374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } attempt: 1
I20260809 23:16:52.442232 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.442039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:43173 (local address 127.0.0.1:58858)
0809 23:16:52.442182 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.442212 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260809 23:16:52.442230 25499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/data/instance:
uuid: "ac16e266004b4b959ccdf6b33b6998e8"
format_stamp: "Formatted at 2026-08-09 23:16:52 on dist-test-slave-7vjv"
server_key: "a3eba439f2fd7d49b7d3f0e68b6330f2"
server_key_iv: "0cebfe3be5804e8cfbed383af580f493"
server_key_version: "encryptionkey@0"
I20260809 23:16:52.442597 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.442520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:43173 (local address 127.0.0.1:58862)
0809 23:16:52.442560 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.442582 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 23:16:52.442606 25499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/wal/instance:
uuid: "ac16e266004b4b959ccdf6b33b6998e8"
format_stamp: "Formatted at 2026-08-09 23:16:52 on dist-test-slave-7vjv"
server_key: "a3eba439f2fd7d49b7d3f0e68b6330f2"
server_key_iv: "0cebfe3be5804e8cfbed383af580f493"
server_key_version: "encryptionkey@0"
W20260809 23:16:52.442706 25374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.252:43173: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
I20260809 23:16:52.443810 25499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:16:52.444514 25523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:52.444666 25499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:52.444751 25499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/wal
uuid: "ac16e266004b4b959ccdf6b33b6998e8"
format_stamp: "Formatted at 2026-08-09 23:16:52 on dist-test-slave-7vjv"
server_key: "a3eba439f2fd7d49b7d3f0e68b6330f2"
server_key_iv: "0cebfe3be5804e8cfbed383af580f493"
server_key_version: "encryptionkey@0"
I20260809 23:16:52.444824 25499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:52.464316 25499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:52.465313 25491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } attempt: 2
I20260809 23:16:52.465606 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.465496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:43173 (local address 127.0.0.1:58866)
0809 23:16:52.465541 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.465569 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 23:16:52.465952 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.465845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:43173 (local address 127.0.0.1:58874)
0809 23:16:52.465902 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.465929 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 23:16:52.466056 25491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.252:43173: Network error: Client connection negotiation failed: client connection to 127.2.76.252:43173: connect: Connection refused (error 111)
I20260809 23:16:52.466276 25499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:52.466369 25499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:52.469977 25499 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.252:43173
I20260809 23:16:52.470016 25607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.252:43173 every 8 connection(s)
I20260809 23:16:52.470331 25499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/data/info.pb
I20260809 23:16:52.471110 25610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:52.472422 25610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.252@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:52.479476 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.472965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43546 (local address 127.2.76.254:46517)
0809 23:16:52.473018 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:52.473019 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:52.473189 (+ 170us) server_negotiation.cc:407] Connection header received
0809 23:16:52.473529 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:52.473530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:52.473537 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:52.473581 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:52.474024 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.474417 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.475220 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.475373 (+ 153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.475428 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.477265 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:52.477266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:52.477267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:52.477272 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:52.478301 (+ 1029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.478654 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.478655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.478655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.478705 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.478921 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.478922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.478922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.479020 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:52.479082 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:52.479310 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 23:16:52.479389 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 23:16:52.479643 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.472955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:46517 (local address 127.0.0.1:43546)
0809 23:16:52.473095 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.473122 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:52.473457 (+ 335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:52.473629 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:52.473631 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:52.473820 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:52.473967 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.473970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.474488 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.474489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.475158 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.475160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.475448 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.475449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.475537 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.475956 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:52.475964 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:52.477164 (+ 1200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:52.478378 (+ 1214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.478385 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.478386 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.478578 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.478767 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.478768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.478768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.478858 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.479076 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:52.479083 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:52.479199 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 23:16:52.479289 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 23:16:52.479421 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
I20260809 23:16:52.480131 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 25499
I20260809 23:16:52.480129 25610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:16:52.480232 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786317238298104-2355-0/minicluster-data/master-2/wal/instance
I20260809 23:16:52.480430 2355 external_mini_cluster.cc:1428] Setting key 89c18e13d8d757639df9dacca1491ad8
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.252@KRBTEST.COM for kudu/127.2.76.253@KRBTEST.COM
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.252@KRBTEST.COM
I20260809 23:16:52.486718 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.480338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:45317 (local address 127.0.0.1:58826)
0809 23:16:52.480424 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.480436 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:52.480489 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:52.480607 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:52.480607 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:52.480757 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:52.480811 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.480829 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.481519 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.481520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.482053 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.482055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.482485 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.482486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.482540 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.482833 (+ 293us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:52.482837 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:52.484036 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:52.485766 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.485767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.485767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.485938 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.486151 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.486152 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.486152 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.486199 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.486437 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:52.486438 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:52.486478 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 23:16:52.486558 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 23:16:52.486640 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 23:16:52.486770 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.480386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58826 (local address 127.2.76.253:45317)
0809 23:16:52.480428 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:52.480429 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:52.480467 (+ 38us) server_negotiation.cc:407] Connection header received
0809 23:16:52.480536 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:52.480537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:52.480546 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:52.480603 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:52.480928 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.481438 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.482132 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.482419 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.482481 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.484107 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:52.484109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:52.484109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:52.484115 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:52.485670 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.486025 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.486026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.486026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.486081 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.486240 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.486241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.486241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.486358 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:52.486428 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:52.486600 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 23:16:52.486672 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:16:52.487339 25610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:16:52.489063 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.480759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58878 (local address 127.2.76.252:43173)
0809 23:16:52.480895 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:52.480898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:52.481057 (+ 159us) server_negotiation.cc:407] Connection header received
0809 23:16:52.481079 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:52.481081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:52.481093 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:52.481173 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:52.481727 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.482551 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.483492 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.483893 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.483956 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.486195 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:52.486201 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:52.486201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:52.486207 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:52.488028 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.488382 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.488385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.488385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.488443 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.488589 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.488590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.488590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.488725 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:52.488872 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:52.488915 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 23:16:52.488982 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":90,"threads_started":1}
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.252@KRBTEST.COM for kudu/127.2.76.252@KRBTEST.COM
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:52.494102 25612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.487605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:43173 (local address 127.0.0.1:58884)
0809 23:16:52.487644 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.487653 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:52.487689 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:52.487891 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:52.487892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:52.488076 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:52.488128 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.488130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.488893 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.488894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.489463 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.489466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.489913 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.489914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.489969 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.490247 (+ 278us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:52.490252 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:52.491381 (+ 1129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:52.493178 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.493179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.493180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.493374 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.493571 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.493571 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.493572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.493618 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.493834 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:52.493835 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:52.493868 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 23:16:52.493925 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 23:16:52.494016 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 23:16:52.494206 25616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.487651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58884 (local address 127.2.76.252:43173)
0809 23:16:52.487791 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:52.487792 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:52.487798 (+ 6us) server_negotiation.cc:407] Connection header received
0809 23:16:52.487812 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:52.487812 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:52.487819 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:52.487889 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:52.488207 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.488825 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.489557 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.489854 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.489911 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.491448 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:52.491449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:52.491450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:52.491455 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:52.493101 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.493444 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.493445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.493445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.493504 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.493683 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.493684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.493685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.493779 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:52.493841 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:52.493941 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 23:16:52.494082 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260809 23:16:52.495419 25610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8: Bootstrap starting.
I20260809 23:16:52.496223 25610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:52.496542 25610 log.cc:824] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:52.496816 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.489730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43552 (local address 127.2.76.254:46517)
0809 23:16:52.489769 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:52.489771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:52.489778 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:16:52.489832 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:52.489832 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:52.489842 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:52.489895 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:52.490434 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.490808 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.491642 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.491942 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.492032 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.494155 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:52.494156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:52.494156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:52.494163 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:52.495645 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.496021 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.496022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.496022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.496068 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.496311 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.496312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.496312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.496431 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:52.496565 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:52.496648 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 23:16:52.496718 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:16:52.497305 25610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8: No bootstrap required, opened a new log
I20260809 23:16:52.498917 25610 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } }
I20260809 23:16:52.499019 25610 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:52.499040 25610 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac16e266004b4b959ccdf6b33b6998e8, State: Initialized, Role: FOLLOWER
I20260809 23:16:52.499158 25610 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } }
I20260809 23:16:52.499384 25619 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } } }
I20260809 23:16:52.499441 25610 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:16:52.499480 25619 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 23:16:52.501336 25637 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:16:52.501446 25637 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.253@KRBTEST.COM
I20260809 23:16:52.501947 25499 master_runner.cc:186] Error getting master registration for 127.2.76.254:46517: OK, instance_id { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" instance_seqno: 1786317412180509 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:16:52.503234 25499 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:16:52.505329 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.497387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58836 (local address 127.2.76.253:45317)
0809 23:16:52.497439 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:52.497441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:52.497501 (+ 60us) server_negotiation.cc:407] Connection header received
0809 23:16:52.497553 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:52.497553 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:52.497561 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:52.497619 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:52.497984 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.498582 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.499212 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.499451 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.499517 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.503313 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:52.503314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:52.503315 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:52.503321 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:52.504468 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.504789 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.504790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.504790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.504832 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.504977 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.504977 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.504977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.505053 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:52.505157 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:52.505235 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 23:16:52.505283 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 23:16:52.506623 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:16:52.514602 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.507859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58840 (local address 127.2.76.253:45317)
0809 23:16:52.507888 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:52.507889 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:52.508203 (+ 314us) server_negotiation.cc:407] Connection header received
0809 23:16:52.508284 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:52.508285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:52.508294 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:52.508318 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:52.508974 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.509370 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.510147 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.510322 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.510383 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.511395 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:52.511397 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:52.511397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:52.511404 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:52.512830 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.513315 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.513316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.513316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.513364 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.513556 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.513557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.513557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.513661 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:52.514245 (+ 584us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:52.514451 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 23:16:52.514521 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11}
I20260809 23:16:52.515484 25380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.507973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43558 (local address 127.2.76.254:46517)
0809 23:16:52.508010 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:52.508012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:52.508071 (+ 59us) server_negotiation.cc:407] Connection header received
0809 23:16:52.508181 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:52.508182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:52.508191 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:52.508230 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:52.508959 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.509655 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.510780 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.511064 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.511103 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.512451 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:52.512452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:52.512452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:52.512460 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:52.513984 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.514400 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.514401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.514401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.514452 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.514657 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.514657 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.514658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.514764 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:52.514821 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:52.515317 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 23:16:52.515388 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:16:52.517882 25374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } attempt: 2
I20260809 23:16:52.519863 25616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.508112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58898 (local address 127.2.76.252:43173)
0809 23:16:52.508142 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:52.508144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:52.512251 (+ 4107us) server_negotiation.cc:407] Connection header received
0809 23:16:52.512979 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:52.512980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:52.512990 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:52.513049 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:52.513811 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.514504 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.515676 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.515915 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.515966 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.517128 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:52.517129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:52.517129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:52.517134 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:52.518674 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.519014 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.519015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.519016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.519067 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.519273 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.519274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.519274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.519430 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:52.519514 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:52.519716 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 23:16:52.519789 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for kudu/127.2.76.252@KRBTEST.COM
I20260809 23:16:52.524151 25378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.518134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:43173 (local address 127.0.0.1:58902)
0809 23:16:52.518178 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.518187 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:52.518222 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:52.518355 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:52.518356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:52.518523 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:52.518564 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.518566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.519419 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.519420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.520119 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.520121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.520526 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.520527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.520576 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.520888 (+ 312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:52.520891 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:52.522012 (+ 1121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:52.523225 (+ 1213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.523226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.523226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.523412 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.523564 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.523565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.523565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.523616 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.523851 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:52.523852 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:52.523891 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 23:16:52.523950 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 23:16:52.524044 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 23:16:52.524484 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.518180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58902 (local address 127.2.76.252:43173)
0809 23:16:52.518214 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:52.518215 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:52.518223 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:16:52.518272 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:52.518273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:52.518282 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:52.518342 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:52.518641 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.519340 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.520184 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.520422 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.520480 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.522078 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:52.522079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:52.522079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:52.522084 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:52.523158 (+ 1074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.523473 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.523473 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.523474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.523516 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.523666 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.523666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.523667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.523754 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:52.523823 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:52.524341 (+ 518us) server_negotiation.cc:299] Negotiation successful
0809 23:16:52.524413 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:16:52.525748 25374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01: Bootstrap starting.
I20260809 23:16:52.526365 25374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:52.526633 25374 log.cc:824] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:52.527330 25374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01: No bootstrap required, opened a new log
I20260809 23:16:52.529296 25374 raft_consensus.cc:348] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } }
I20260809 23:16:52.529430 25374 raft_consensus.cc:374] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:52.529464 25374 raft_consensus.cc:729] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b02b1d7819d046cea6f4a4eb9b0f1f01, State: Initialized, Role: FOLLOWER
I20260809 23:16:52.529564 25374 consensus_queue.cc:260] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } }
I20260809 23:16:52.529793 25653 sys_catalog.cc:455] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } } }
I20260809 23:16:52.529824 25374 sys_catalog.cc:564] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:16:52.529932 25653 sys_catalog.cc:458] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 23:16:52.531759 25676 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:16:52.531806 25676 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:16:52.532076 25256 master_runner.cc:186] Error getting master registration for 127.2.76.253:45317: OK, instance_id { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" instance_seqno: 1786317412323963 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:16:52.534279 25256 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:16:52.557197 25491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } attempt: 3
Aug 09 23:16:52 dist-test-slave-7vjv krb5kdc[25237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317412, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.253@KRBTEST.COM for kudu/127.2.76.252@KRBTEST.COM
I20260809 23:16:52.564019 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.557427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:43173 (local address 127.0.0.1:58912)
0809 23:16:52.557497 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:52.557507 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:52.557550 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:52.557762 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:52.557763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:52.557929 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:52.557968 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.557969 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.558815 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.558816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.559503 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:52.559507 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.559955 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.559956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:52.560015 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.560406 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:52.560411 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:52.561603 (+ 1192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:52.563109 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.563110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.563110 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.563286 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.563482 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:52.563482 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:52.563483 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:52.563534 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:52.563768 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:52.563769 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:52.563802 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 23:16:52.563853 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 23:16:52.563927 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 23:16:52.564347 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:52.557608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58912 (local address 127.2.76.252:43173)
0809 23:16:52.557638 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:52.557640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:52.557646 (+ 6us) server_negotiation.cc:407] Connection header received
0809 23:16:52.557664 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:52.557665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:52.557673 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:52.557729 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:52.558070 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.558729 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.559606 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:52.559857 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:52.559910 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:52.561688 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:52.561689 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:52.561690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:52.561695 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:52.563026 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.563354 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.563355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.563356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.563403 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:52.563593 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:52.563594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:52.563594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:52.563700 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:52.563747 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:52.564207 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 23:16:52.564273 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 23:16:52.565642 25491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1: Bootstrap starting.
I20260809 23:16:52.566282 25491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:52.566568 25491 log.cc:824] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:52.568636 25491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1: No bootstrap required, opened a new log
I20260809 23:16:52.570478 25491 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } }
I20260809 23:16:52.570609 25491 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:52.570649 25491 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6d1acc11454d96b19a84d51199dbc1, State: Initialized, Role: FOLLOWER
I20260809 23:16:52.570756 25491 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } }
I20260809 23:16:52.571050 25491 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 23:16:52.571978 25415 tablet.cc:2369] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:16:52.572736 25705 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:16:52.572788 25705 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:16:52.572929 25679 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } } }
I20260809 23:16:52.573004 25679 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:16:52.574290 25388 master_runner.cc:263] No leader master found from master fc6d1acc11454d96b19a84d51199dbc1
I20260809 23:16:52.574348 25388 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 23:16:52.678609 25290 tablet.cc:2369] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:16:52.716867 25530 tablet.cc:2369] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:16:52.720705 25619 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:16:52.720791 25619 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } }
I20260809 23:16:52.721014 25619 leader_election.cc:288] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b02b1d7819d046cea6f4a4eb9b0f1f01 (127.2.76.254:46517), fc6d1acc11454d96b19a84d51199dbc1 (127.2.76.253:45317)
I20260809 23:16:52.721190 25331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac16e266004b4b959ccdf6b33b6998e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" is_pre_election: true
I20260809 23:16:52.721244 25448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac16e266004b4b959ccdf6b33b6998e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6d1acc11454d96b19a84d51199dbc1" is_pre_election: true
I20260809 23:16:52.721331 25331 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac16e266004b4b959ccdf6b33b6998e8 in term 0.
I20260809 23:16:52.721395 25448 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac16e266004b4b959ccdf6b33b6998e8 in term 0.
I20260809 23:16:52.721500 25526 leader_election.cc:302] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac16e266004b4b959ccdf6b33b6998e8, b02b1d7819d046cea6f4a4eb9b0f1f01; no voters:
I20260809 23:16:52.721684 25619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:16:52.721737 25619 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:16:52.721751 25619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:52.722384 25619 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } }
I20260809 23:16:52.722500 25619 leader_election.cc:288] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [CANDIDATE]: Term 1 election: Requested vote from peers b02b1d7819d046cea6f4a4eb9b0f1f01 (127.2.76.254:46517), fc6d1acc11454d96b19a84d51199dbc1 (127.2.76.253:45317)
I20260809 23:16:52.722615 25331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac16e266004b4b959ccdf6b33b6998e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01"
I20260809 23:16:52.722697 25331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:52.722723 25448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ac16e266004b4b959ccdf6b33b6998e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6d1acc11454d96b19a84d51199dbc1"
I20260809 23:16:52.722852 25448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:52.723465 25331 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac16e266004b4b959ccdf6b33b6998e8 in term 1.
I20260809 23:16:52.723627 25526 leader_election.cc:302] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac16e266004b4b959ccdf6b33b6998e8, b02b1d7819d046cea6f4a4eb9b0f1f01; no voters:
I20260809 23:16:52.723727 25619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:16:52.723948 25619 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [term 1 LEADER]: Becoming Leader. State: Replica: ac16e266004b4b959ccdf6b33b6998e8, State: Running, Role: LEADER
I20260809 23:16:52.724054 25619 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } }
I20260809 23:16:52.724200 25448 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac16e266004b4b959ccdf6b33b6998e8 in term 1.
I20260809 23:16:52.724478 25707 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac16e266004b4b959ccdf6b33b6998e8. Latest consensus state: current_term: 1 leader_uuid: "ac16e266004b4b959ccdf6b33b6998e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } } }
I20260809 23:16:52.724574 25707 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:52.724749 25711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:16:52.724906 25711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:16:52.726008 25448 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1 [term 1 FOLLOWER]: Refusing update from remote peer ac16e266004b4b959ccdf6b33b6998e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:16:52.726045 25331 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01 [term 1 FOLLOWER]: Refusing update from remote peer ac16e266004b4b959ccdf6b33b6998e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:16:52.726310 25619 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:16:52.726425 25707 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:16:52.727396 25679 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac16e266004b4b959ccdf6b33b6998e8. Latest consensus state: current_term: 1 leader_uuid: "ac16e266004b4b959ccdf6b33b6998e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } } }
I20260809 23:16:52.727406 25653 sys_catalog.cc:455] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac16e266004b4b959ccdf6b33b6998e8. Latest consensus state: current_term: 1 leader_uuid: "ac16e266004b4b959ccdf6b33b6998e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } } }
I20260809 23:16:52.727485 25679 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:16:52.727488 25653 sys_catalog.cc:458] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:16:52.728361 25619 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac16e266004b4b959ccdf6b33b6998e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } } }
I20260809 23:16:52.728433 25619 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:52.728745 25679 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac16e266004b4b959ccdf6b33b6998e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } } }
I20260809 23:16:52.728819 25679 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:16:52.728811 25711 catalog_manager.cc:1347] Generated new cluster ID: c30bee338ba64c86ad52894b9382167f
I20260809 23:16:52.728955 25711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:16:52.729110 25707 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ac16e266004b4b959ccdf6b33b6998e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } } }
I20260809 23:16:52.729171 25707 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8 [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:52.729301 25720 mvcc.cc:205] Tried to move back new op lower bound from 7316756122523910144 to 7316756122518269952. Current Snapshot: MvccSnapshot[applied={T|T < 7316756122523910144 or (T in {7316756122523910144})}]
I20260809 23:16:52.729475 25653 sys_catalog.cc:455] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ac16e266004b4b959ccdf6b33b6998e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b02b1d7819d046cea6f4a4eb9b0f1f01" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 46517 } } peers { permanent_uuid: "fc6d1acc11454d96b19a84d51199dbc1" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 45317 } } peers { permanent_uuid: "ac16e266004b4b959ccdf6b33b6998e8" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 43173 } } }
I20260809 23:16:52.729573 25653 sys_catalog.cc:458] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:16:52.729852 25715 mvcc.cc:205] Tried to move back new op lower bound from 7316756122523910144 to 7316756122518269952. Current Snapshot: MvccSnapshot[applied={T|T < 7316756122523910144 or (T in {7316756122523910144})}]
I20260809 23:16:52.758160 25711 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:16:52.758644 25711 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:16:52.765025 25711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac16e266004b4b959ccdf6b33b6998e8: Generated new TSK 0
I20260809 23:16:52.765173 25711 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:16:53.532250 25676 catalog_manager.cc:1259] Loaded cluster ID: c30bee338ba64c86ad52894b9382167f
I20260809 23:16:53.532327 25676 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01: loading cluster ID for follower catalog manager: success
I20260809 23:16:53.533277 25676 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01: acquiring CA information for follower catalog manager: success
I20260809 23:16:53.533658 25676 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b02b1d7819d046cea6f4a4eb9b0f1f01: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:16:53.573158 25705 catalog_manager.cc:1259] Loaded cluster ID: c30bee338ba64c86ad52894b9382167f
I20260809 23:16:53.573210 25705 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1: loading cluster ID for follower catalog manager: success
I20260809 23:16:53.573877 25705 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1: acquiring CA information for follower catalog manager: success
I20260809 23:16:53.574093 25705 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fc6d1acc11454d96b19a84d51199dbc1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:16:53.819922 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 25256
I20260809 23:16:53.835103 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 25388
I20260809 23:16:53.846621 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 25499
2026-08-09T23:16:53Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3020 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:16:53 dist-test-slave-7vjv krb5kdc[25746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:16:53 dist-test-slave-7vjv krb5kdc[25746](info): set up 2 sockets
Aug 09 23:16:53 dist-test-slave-7vjv krb5kdc[25746](info): commencing operation
krb5kdc: starting...
W20260809 23:16:54.980036 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:16:55 dist-test-slave-7vjv krb5kdc[25746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:16:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:16:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:16:55.084156 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:39905
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:35665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:39905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:55.153810 25762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:55.154001 25762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:55.154037 25762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:55.155122 25762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:16:55.155184 25762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:55.155206 25762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:55.155233 25762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:16:55.155262 25762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:16:55.156577 25762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:35665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:39905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:39905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:55.156829 25762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:55.157023 25762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:55.161186 25773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:55.161229 25770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:55.161278 25762 server_base.cc:1034] running on GCE node
W20260809 23:16:55.161226 25768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:55.161749 25762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:55.161960 25762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:55.163093 25762 hybrid_clock.cc:648] HybridClock initialized: now 1786317415163082 us; error 26 us; skew 500 ppm
Aug 09 23:16:55 dist-test-slave-7vjv krb5kdc[25746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317415, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:55.165549 25762 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:16:55.165974 25762 webserver.cc:460] Webserver started at http://127.2.76.254:42623/ using document root <none> and password file <none>
I20260809 23:16:55.166168 25762 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:55.166208 25762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:55.166347 25762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:16:55.167241 25762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "b2c815fd042840b5b499f4847e818d6f"
format_stamp: "Formatted at 2026-08-09 23:16:55 on dist-test-slave-7vjv"
server_key: "1cd96286472501e2b4bd09b8ca172511"
server_key_iv: "99b00d0d0255dda623f1ea822b2714e8"
server_key_version: "encryptionkey@0"
I20260809 23:16:55.167557 25762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "b2c815fd042840b5b499f4847e818d6f"
format_stamp: "Formatted at 2026-08-09 23:16:55 on dist-test-slave-7vjv"
server_key: "1cd96286472501e2b4bd09b8ca172511"
server_key_iv: "99b00d0d0255dda623f1ea822b2714e8"
server_key_version: "encryptionkey@0"
I20260809 23:16:55.168788 25762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:16:55.169384 25783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:55.169555 25762 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:16:55.169621 25762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "b2c815fd042840b5b499f4847e818d6f"
format_stamp: "Formatted at 2026-08-09 23:16:55 on dist-test-slave-7vjv"
server_key: "1cd96286472501e2b4bd09b8ca172511"
server_key_iv: "99b00d0d0255dda623f1ea822b2714e8"
server_key_version: "encryptionkey@0"
I20260809 23:16:55.169699 25762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:55.191385 25762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:55.193281 25762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:55.193375 25762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:55.196923 25762 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:39905
I20260809 23:16:55.196969 25870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:39905 every 8 connection(s)
I20260809 23:16:55.197255 25762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:16:55.197999 25871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:55.198024 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 25762
I20260809 23:16:55.198112 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:16:55.198287 2355 external_mini_cluster.cc:1428] Setting key 36f348ac6d0f2bc89e972392e03d0f3b
I20260809 23:16:55.200858 25871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f: Bootstrap starting.
I20260809 23:16:55.201537 25871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:55.201795 25871 log.cc:824] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:55.202519 25871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f: No bootstrap required, opened a new log
Aug 09 23:16:55 dist-test-slave-7vjv krb5kdc[25746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:55.204895 25871 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c815fd042840b5b499f4847e818d6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39905 } }
I20260809 23:16:55.205044 25871 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:55.205065 25871 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c815fd042840b5b499f4847e818d6f, State: Initialized, Role: FOLLOWER
I20260809 23:16:55.205260 25871 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c815fd042840b5b499f4847e818d6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39905 } }
I20260809 23:16:55.205377 25871 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:16:55.205410 25871 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:16:55.205459 25871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:55.206130 25871 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c815fd042840b5b499f4847e818d6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39905 } }
I20260809 23:16:55.206233 25871 leader_election.cc:302] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2c815fd042840b5b499f4847e818d6f; no voters:
I20260809 23:16:55.206379 25871 leader_election.cc:288] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:16:55.206446 25880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:16:55.206607 25871 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:16:55.206619 25880 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [term 1 LEADER]: Becoming Leader. State: Replica: b2c815fd042840b5b499f4847e818d6f, State: Running, Role: LEADER
I20260809 23:16:55.206745 25880 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c815fd042840b5b499f4847e818d6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39905 } }
I20260809 23:16:55.207036 25880 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2c815fd042840b5b499f4847e818d6f. Latest consensus state: current_term: 1 leader_uuid: "b2c815fd042840b5b499f4847e818d6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c815fd042840b5b499f4847e818d6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39905 } } }
I20260809 23:16:55.207046 25881 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2c815fd042840b5b499f4847e818d6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2c815fd042840b5b499f4847e818d6f" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39905 } } }
I20260809 23:16:55.207125 25880 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:55.207177 25881 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:55.207899 25891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:16:55.208447 25875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:55.198707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34478 (local address 127.2.76.254:39905)
0809 23:16:55.198860 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:55.198867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:55.198898 (+ 31us) server_negotiation.cc:407] Connection header received
0809 23:16:55.199273 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:55.199279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:55.199316 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:55.199409 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:55.200176 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:55.201004 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:55.201932 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:55.202309 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:55.202384 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:55.204710 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:55.204714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:55.204714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:55.204724 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:55.206580 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:55.206994 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:55.206997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:55.206997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:55.207110 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:55.207395 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:55.207396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:55.207396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:55.207694 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:55.207839 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:55.208208 (+ 369us) server_negotiation.cc:299] Negotiation successful
0809 23:16:55.208327 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260809 23:16:55.208778 25891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:16:55.209203 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:16:55.211071 25891 catalog_manager.cc:1347] Generated new cluster ID: 5486fd1a8be2415f92af71539869f91d
I20260809 23:16:55.211122 25891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:16:55.218902 25891 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:16:55.219503 25891 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:16:55.226056 25891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2c815fd042840b5b499f4847e818d6f: Generated new TSK 0
I20260809 23:16:55.226195 25891 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:16:55.262616 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 25762
2026-08-09T23:16:55Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1425 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:16:55 dist-test-slave-7vjv krb5kdc[25913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:16:55 dist-test-slave-7vjv krb5kdc[25913](info): set up 2 sockets
Aug 09 23:16:55 dist-test-slave-7vjv krb5kdc[25913](info): commencing operation
krb5kdc: starting...
W20260809 23:16:56.421739 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:16:56 dist-test-slave-7vjv krb5kdc[25913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:16:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:16:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:16:56.530107 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45069
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:45069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:56.602828 25936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:56.602981 25936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:56.603005 25936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:56.603964 25936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:16:56.604000 25936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:56.604017 25936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:56.604063 25936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:16:56.604080 25936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:16:56.605329 25936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:45069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:56.605549 25936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:56.605748 25936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:56.609936 25943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:56.609974 25944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:56.609992 25949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:56.610332 25936 server_base.cc:1034] running on GCE node
I20260809 23:16:56.610580 25936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:56.610864 25936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:56.612000 25936 hybrid_clock.cc:648] HybridClock initialized: now 1786317416611982 us; error 34 us; skew 500 ppm
Aug 09 23:16:56 dist-test-slave-7vjv krb5kdc[25913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317416, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:56.618100 25936 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:16:56.618570 25936 webserver.cc:460] Webserver started at http://127.2.76.254:41625/ using document root <none> and password file <none>
I20260809 23:16:56.618786 25936 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:56.618827 25936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:56.618955 25936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:16:56.619887 25936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "ed5802d1b2df47ceb62dd6d94e2283cc"
format_stamp: "Formatted at 2026-08-09 23:16:56 on dist-test-slave-7vjv"
server_key: "cc52728dc0a64143798ff41d2c4040ad"
server_key_iv: "d6d9774defa2c8f1738fdcfeac375ef9"
server_key_version: "encryptionkey@0"
I20260809 23:16:56.620214 25936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "ed5802d1b2df47ceb62dd6d94e2283cc"
format_stamp: "Formatted at 2026-08-09 23:16:56 on dist-test-slave-7vjv"
server_key: "cc52728dc0a64143798ff41d2c4040ad"
server_key_iv: "d6d9774defa2c8f1738fdcfeac375ef9"
server_key_version: "encryptionkey@0"
I20260809 23:16:56.621392 25936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:16:56.622051 25957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:56.622249 25936 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:16:56.622331 25936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "ed5802d1b2df47ceb62dd6d94e2283cc"
format_stamp: "Formatted at 2026-08-09 23:16:56 on dist-test-slave-7vjv"
server_key: "cc52728dc0a64143798ff41d2c4040ad"
server_key_iv: "d6d9774defa2c8f1738fdcfeac375ef9"
server_key_version: "encryptionkey@0"
I20260809 23:16:56.622411 25936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:56.636960 25936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:56.638397 25936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:56.638541 25936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:56.644615 25936 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:45069
I20260809 23:16:56.644665 26044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:45069 every 8 connection(s)
I20260809 23:16:56.644930 25936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:16:56.645725 26045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:56.647655 26045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: Bootstrap starting.
I20260809 23:16:56.648247 26045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:56.648522 26045 log.cc:824] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:56.649150 26045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: No bootstrap required, opened a new log
I20260809 23:16:56.650489 26045 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } }
I20260809 23:16:56.650632 26045 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:56.650673 26045 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed5802d1b2df47ceb62dd6d94e2283cc, State: Initialized, Role: FOLLOWER
I20260809 23:16:56.650813 26045 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } }
I20260809 23:16:56.650883 26045 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:16:56.650921 26045 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:16:56.650988 26045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:56.651697 26045 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } }
I20260809 23:16:56.651806 26045 leader_election.cc:302] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed5802d1b2df47ceb62dd6d94e2283cc; no voters:
I20260809 23:16:56.651973 26045 leader_election.cc:288] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:16:56.651996 26049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:16:56.652150 26049 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 1 LEADER]: Becoming Leader. State: Replica: ed5802d1b2df47ceb62dd6d94e2283cc, State: Running, Role: LEADER
I20260809 23:16:56.652241 26045 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:16:56.652277 26049 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } }
I20260809 23:16:56.652673 26054 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed5802d1b2df47ceb62dd6d94e2283cc. Latest consensus state: current_term: 1 leader_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } } }
I20260809 23:16:56.652767 26054 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:56.653020 26051 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } } }
I20260809 23:16:56.653105 26051 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:56.653210 26066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:16:56.653723 26066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:16:56.654153 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 25936
I20260809 23:16:56.654246 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:16:56.654415 2355 external_mini_cluster.cc:1428] Setting key e67858a7ea8c6b6953a5de37066a6a87
I20260809 23:16:56.655666 26066 catalog_manager.cc:1347] Generated new cluster ID: 78a50c2b4d754a8fa383180f92ab164e
I20260809 23:16:56.655720 26066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:16:56.660137 26066 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:16:56.660730 26066 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 23:16:56 dist-test-slave-7vjv krb5kdc[25913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:56.665851 26082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:56.654809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54004 (local address 127.2.76.254:45069)
0809 23:16:56.654949 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:56.654957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:56.656330 (+ 1373us) server_negotiation.cc:407] Connection header received
0809 23:16:56.656454 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:56.656460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:56.656510 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:56.656646 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:56.657261 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:56.658448 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:56.659419 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:56.660031 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:56.660166 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:56.662660 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:56.662663 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:56.662664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:56.662674 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:56.664403 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:56.664764 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:56.664766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:56.664766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:56.664856 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:56.665057 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:56.665057 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:56.665057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:56.665264 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:56.665394 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:56.665544 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 23:16:56.665661 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
I20260809 23:16:56.667164 26066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: Generated new TSK 0
I20260809 23:16:56.667306 26066 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:16:56.695847 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45069
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:56.802035 26088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:56.802269 26088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:56.802297 26088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:56.803743 26088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:56.803821 26088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:56.803879 26088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:16:56.805651 26088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:56.805971 26088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:56.806212 26088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:56.811631 26094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:56.811642 26098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:56.811612 26095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:56.811875 26088 server_base.cc:1034] running on GCE node
I20260809 23:16:56.812065 26088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:56.812307 26088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:56.813462 26088 hybrid_clock.cc:648] HybridClock initialized: now 1786317416813457 us; error 40 us; skew 500 ppm
Aug 09 23:16:56 dist-test-slave-7vjv krb5kdc[25913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317416, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:56.815758 26088 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:16:56.816148 26088 webserver.cc:460] Webserver started at http://127.2.76.193:36953/ using document root <none> and password file <none>
I20260809 23:16:56.816321 26088 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:56.816357 26088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:56.816442 26088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:16:56.817265 26088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "a2eb4bd3849c484b9b62c1740463113d"
format_stamp: "Formatted at 2026-08-09 23:16:56 on dist-test-slave-7vjv"
server_key: "5c4171e409a075bf70b6580b93c95f90"
server_key_iv: "528ec0dc047ad3964809d542ccec23b7"
server_key_version: "encryptionkey@0"
I20260809 23:16:56.817531 26088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "a2eb4bd3849c484b9b62c1740463113d"
format_stamp: "Formatted at 2026-08-09 23:16:56 on dist-test-slave-7vjv"
server_key: "5c4171e409a075bf70b6580b93c95f90"
server_key_iv: "528ec0dc047ad3964809d542ccec23b7"
server_key_version: "encryptionkey@0"
I20260809 23:16:56.818681 26088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:16:56.819380 26108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:56.819522 26088 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:56.819604 26088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "a2eb4bd3849c484b9b62c1740463113d"
format_stamp: "Formatted at 2026-08-09 23:16:56 on dist-test-slave-7vjv"
server_key: "5c4171e409a075bf70b6580b93c95f90"
server_key_iv: "528ec0dc047ad3964809d542ccec23b7"
server_key_version: "encryptionkey@0"
I20260809 23:16:56.819651 26088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:56.832494 26088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:56.833930 26088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:56.834020 26088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:56.834158 26088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:56.834411 26088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:56.834440 26088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:56.834457 26088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:56.834497 26088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:56.839533 26088 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:44045
I20260809 23:16:56.839612 26285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:44045 every 8 connection(s)
I20260809 23:16:56.839872 26088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:16:56.840610 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 26088
I20260809 23:16:56.840716 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:16:56.840898 2355 external_mini_cluster.cc:1428] Setting key 766b5bce238a5f955a9c7221b9e375ba
Aug 09 23:16:56 dist-test-slave-7vjv krb5kdc[25913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317416, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:56.849309 26290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:56.840491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:57473)
0809 23:16:56.840688 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:56.840735 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:56.841295 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:56.841548 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:56.841551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:56.842096 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:56.842242 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:56.842246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:56.842853 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:56.842854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:56.843931 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:56.843935 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:56.844393 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:56.844394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:56.844481 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:56.844981 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:56.844988 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:56.846265 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:56.848178 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:56.848187 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:56.848192 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:56.848376 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:56.848581 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:56.848582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:56.848582 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:56.848668 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:56.848921 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:56.848924 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:56.849019 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 23:16:56.849087 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 23:16:56.849181 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
I20260809 23:16:56.849324 26082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:56.840516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:57473 (local address 127.2.76.254:45069)
0809 23:16:56.840556 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:56.840558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:56.840823 (+ 265us) server_negotiation.cc:407] Connection header received
0809 23:16:56.841396 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:56.841397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:56.841409 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:56.841465 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:56.842332 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:56.842760 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:56.844026 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:56.844310 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:56.844375 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:56.846370 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:56.846372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:56.846372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:56.846380 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:56.848052 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:56.848447 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:56.848448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:56.848449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:56.848516 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:56.848730 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:56.848731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:56.848731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:56.848857 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:56.848927 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:56.849137 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 23:16:56.849217 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:16:56.849926 26287 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45069
I20260809 23:16:56.850004 26287 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
I20260809 23:16:56.850169 26287 heartbeater.cc:511] Master 127.2.76.254:45069 requested a full tablet report, sending...
I20260809 23:16:56.850670 25987 ts_manager.cc:194] Registered new tserver with Master: a2eb4bd3849c484b9b62c1740463113d (127.2.76.193:44045)
I20260809 23:16:56.851251 25987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:57473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:16:56.864972 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45069
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:56.935246 26299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:56.935459 26299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:56.935504 26299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:56.936605 26299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:56.936663 26299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:56.936705 26299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:16:56.938031 26299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:56.938261 26299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:56.938460 26299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:56.942904 26306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:56.942947 26309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:56.942903 26307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:56.943157 26299 server_base.cc:1034] running on GCE node
I20260809 23:16:56.943382 26299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:56.943593 26299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:56.944749 26299 hybrid_clock.cc:648] HybridClock initialized: now 1786317416944740 us; error 33 us; skew 500 ppm
Aug 09 23:16:56 dist-test-slave-7vjv krb5kdc[25913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317416, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:56.947052 26299 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:16:56.947476 26299 webserver.cc:460] Webserver started at http://127.2.76.194:32929/ using document root <none> and password file <none>
I20260809 23:16:56.947679 26299 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:56.947741 26299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:56.947865 26299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:16:56.948797 26299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "939dac5520824a25bbb09b7375f14c6e"
format_stamp: "Formatted at 2026-08-09 23:16:56 on dist-test-slave-7vjv"
server_key: "1381b3c82653cdf86eab147bd59087f5"
server_key_iv: "0828a7865feb6f6a337551ecb12c289b"
server_key_version: "encryptionkey@0"
I20260809 23:16:56.949124 26299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "939dac5520824a25bbb09b7375f14c6e"
format_stamp: "Formatted at 2026-08-09 23:16:56 on dist-test-slave-7vjv"
server_key: "1381b3c82653cdf86eab147bd59087f5"
server_key_iv: "0828a7865feb6f6a337551ecb12c289b"
server_key_version: "encryptionkey@0"
I20260809 23:16:56.950300 26299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:16:56.950917 26321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:56.951078 26299 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:16:56.951164 26299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "939dac5520824a25bbb09b7375f14c6e"
format_stamp: "Formatted at 2026-08-09 23:16:56 on dist-test-slave-7vjv"
server_key: "1381b3c82653cdf86eab147bd59087f5"
server_key_iv: "0828a7865feb6f6a337551ecb12c289b"
server_key_version: "encryptionkey@0"
I20260809 23:16:56.951241 26299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:56.971200 26299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:56.972729 26299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:56.972878 26299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:56.973104 26299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:56.973393 26299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:56.973444 26299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:56.973475 26299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:56.973500 26299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:56.978963 26299 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:36437
I20260809 23:16:56.979018 26497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:36437 every 8 connection(s)
I20260809 23:16:56.979298 26299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:16:56 dist-test-slave-7vjv krb5kdc[25913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317416, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:56.987466 26082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:56.979968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:48133 (local address 127.2.76.254:45069)
0809 23:16:56.980030 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:56.980032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:56.980270 (+ 238us) server_negotiation.cc:407] Connection header received
0809 23:16:56.980721 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:56.980722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:56.980735 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:56.980789 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:56.981630 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:56.982092 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:56.982972 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:56.983170 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:56.983237 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:56.985225 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:56.985227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:56.985227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:56.985234 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:56.986362 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:56.986701 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:56.986701 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:56.986702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:56.986750 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:56.986941 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:56.986942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:56.986942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:56.987063 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:56.987114 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:56.987335 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 23:16:56.987403 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 23:16:56.987613 26504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:56.979934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:48133)
0809 23:16:56.980158 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:56.980189 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:56.980615 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:56.980856 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:56.980859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:56.981377 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:56.981533 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:56.981537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:56.982180 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:56.982181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:56.982882 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:56.982884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:56.983258 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:56.983259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:56.983366 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:56.983816 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:56.983823 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:56.985140 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:56.986428 (+ 1288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:56.986434 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:56.986435 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:56.986640 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:56.986805 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:56.986805 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:56.986806 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:56.986892 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:56.987127 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:56.987132 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:56.987230 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 23:16:56.987306 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 23:16:56.987414 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":96,"threads_started":1}
I20260809 23:16:56.988157 26498 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45069
I20260809 23:16:56.988263 26498 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:56.988433 26498 heartbeater.cc:511] Master 127.2.76.254:45069 requested a full tablet report, sending...
I20260809 23:16:56.988811 25987 ts_manager.cc:194] Registered new tserver with Master: 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437)
I20260809 23:16:56.989161 25987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:48133
I20260809 23:16:56.989161 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 26299
I20260809 23:16:56.989266 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:16:56.989446 2355 external_mini_cluster.cc:1428] Setting key 39ab99e20c79e7d244813e51ffbaaddf
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:16:57.018157 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45069
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:57.092682 26508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:57.092844 26508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:57.092862 26508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:57.093863 26508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:57.093902 26508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:57.093952 26508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:16:57.095374 26508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:57.095618 26508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:57.095834 26508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:57.100376 26516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:57.100482 26508 server_base.cc:1034] running on GCE node
W20260809 23:16:57.100366 26519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:57.100364 26517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:57.100797 26508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:57.101019 26508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:57.102166 26508 hybrid_clock.cc:648] HybridClock initialized: now 1786317417102144 us; error 32 us; skew 500 ppm
Aug 09 23:16:57 dist-test-slave-7vjv krb5kdc[25913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317417, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:57.104315 26508 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:16:57.104720 26508 webserver.cc:460] Webserver started at http://127.2.76.195:33771/ using document root <none> and password file <none>
I20260809 23:16:57.104912 26508 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:57.104951 26508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:57.105082 26508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:16:57.106010 26508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "b7f70a187e174dd28553c73cccc1b7d1"
format_stamp: "Formatted at 2026-08-09 23:16:57 on dist-test-slave-7vjv"
server_key: "ec208ad32806b249ad5d693c0a313649"
server_key_iv: "c25f2c01b69e70eefe5785f95fbf345e"
server_key_version: "encryptionkey@0"
I20260809 23:16:57.106346 26508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "b7f70a187e174dd28553c73cccc1b7d1"
format_stamp: "Formatted at 2026-08-09 23:16:57 on dist-test-slave-7vjv"
server_key: "ec208ad32806b249ad5d693c0a313649"
server_key_iv: "c25f2c01b69e70eefe5785f95fbf345e"
server_key_version: "encryptionkey@0"
I20260809 23:16:57.107519 26508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:16:57.108192 26529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:57.108351 26508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:57.108439 26508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "b7f70a187e174dd28553c73cccc1b7d1"
format_stamp: "Formatted at 2026-08-09 23:16:57 on dist-test-slave-7vjv"
server_key: "ec208ad32806b249ad5d693c0a313649"
server_key_iv: "c25f2c01b69e70eefe5785f95fbf345e"
server_key_version: "encryptionkey@0"
I20260809 23:16:57.108517 26508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:57.146826 26508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:57.148335 26508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:57.148478 26508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:57.148669 26508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:57.148931 26508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:57.148979 26508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:57.149021 26508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:57.149045 26508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:57.154982 26508 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:36841
I20260809 23:16:57.155032 26719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:36841 every 8 connection(s)
I20260809 23:16:57.155297 26508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:16:57 dist-test-slave-7vjv krb5kdc[25913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317417, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:57.162680 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 26508
I20260809 23:16:57.162761 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:16:57.162928 2355 external_mini_cluster.cc:1428] Setting key c60aa0f9022c986387774316201b1c63
I20260809 23:16:57.163334 26082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.155885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:41059 (local address 127.2.76.254:45069)
0809 23:16:57.155942 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:57.155944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:57.156143 (+ 199us) server_negotiation.cc:407] Connection header received
0809 23:16:57.156534 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:57.156535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:57.156544 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:57.156594 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:57.157394 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.157856 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.158732 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.158889 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.158933 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.160840 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:57.160841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:57.160842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:57.160848 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:57.162000 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:57.162334 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:57.162334 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:57.162335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:57.162397 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:57.162672 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:57.162673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:57.162673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:57.162798 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:57.162877 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:57.163164 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 23:16:57.163253 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 23:16:57.163460 26724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.155868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:41059)
0809 23:16:57.156029 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:57.156065 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:57.156459 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:57.156648 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:57.156651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:57.157157 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:57.157323 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.157327 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.157940 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.157941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.158667 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.158669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.158952 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.158953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.159039 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.159465 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:57.159479 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:57.160757 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:57.162066 (+ 1309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:57.162072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:57.162073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:57.162263 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:57.162457 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:57.162457 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:57.162458 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:57.162605 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:57.162869 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:57.162877 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:57.163012 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 23:16:57.163126 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 23:16:57.163267 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260809 23:16:57.163885 26721 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45069
I20260809 23:16:57.163985 26721 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:57.164177 26721 heartbeater.cc:511] Master 127.2.76.254:45069 requested a full tablet report, sending...
I20260809 23:16:57.164567 25987 ts_manager.cc:194] Registered new tserver with Master: b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841)
I20260809 23:16:57.164913 25987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:41059
I20260809 23:16:57.174587 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:16:57.180775 26082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.175725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54014 (local address 127.2.76.254:45069)
0809 23:16:57.175782 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:57.175784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:57.175854 (+ 70us) server_negotiation.cc:407] Connection header received
0809 23:16:57.175885 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:57.175885 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:57.175892 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:57.175931 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:57.176555 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.176998 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.177656 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.177820 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.177868 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.178898 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:57.178899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:57.178899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:57.178903 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:57.179900 (+ 997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:57.180245 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:57.180246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:57.180246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:57.180292 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:57.180427 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:57.180428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:57.180428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:57.180524 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:57.180588 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:57.180653 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 23:16:57.180704 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:16:57.182206 25987 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:16:57.183645 25987 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:16:57.189728 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.185795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:36841 (local address 127.0.0.1:56824)
0809 23:16:57.185880 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:57.185913 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:57.185968 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:57.186327 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:57.186330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:57.186331 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:57.186392 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.186394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.187429 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.187429 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.188720 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.188723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.189393 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.189394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.189492 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.189522 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 23:16:57.189589 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 23:16:57.189647 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":36,"thread_start_us":48,"threads_started":1}
I20260809 23:16:57.189814 26743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.186290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43640 (local address 127.2.76.193:44045)
0809 23:16:57.186519 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:57.186522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:57.186531 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:16:57.186565 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:57.186567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:57.186580 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:57.186787 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:57.186859 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.187699 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.188916 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.189427 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.189501 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.189561 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:57.189667 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 23:16:57.189723 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":94,"threads_started":1}
I20260809 23:16:57.189989 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.185811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:44045 (local address 127.0.0.1:43640)
0809 23:16:57.186046 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:57.186056 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:57.186105 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:57.186675 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:57.186676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:57.186678 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:57.186723 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.186724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.187804 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.187805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.188833 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.188835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.189518 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.189519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.189583 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.189587 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:16:57.189652 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 23:16:57.189693 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":27,"spinlock_wait_cycles":768,"thread_start_us":85,"threads_started":1}
I20260809 23:16:57.190096 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.186028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56824 (local address 127.2.76.195:36841)
0809 23:16:57.186175 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:57.186180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:57.186189 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:16:57.186219 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:57.186221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:57.186234 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:57.186310 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:57.186455 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.187352 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.188812 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.189307 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.189365 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.189398 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:57.189983 (+ 585us) server_negotiation.cc:299] Negotiation successful
0809 23:16:57.190025 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260809 23:16:57.190552 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.185813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:36437 (local address 127.0.0.1:48806)
0809 23:16:57.186152 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:57.186159 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:57.186199 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:57.186591 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:57.186592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:57.186594 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:57.186629 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.186631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.187364 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.187365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.188721 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.188723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.190313 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.190314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.190380 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.190384 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:16:57.190450 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 23:16:57.190490 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":44,"thread_start_us":76,"threads_started":1}
I20260809 23:16:57.190694 26742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.186059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48806 (local address 127.2.76.194:36437)
0809 23:16:57.186237 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:57.186248 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:57.186256 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:16:57.186315 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:57.186317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:57.186331 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:57.186512 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:57.186688 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.187272 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.189432 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.190225 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.190296 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.190350 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:57.190463 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 23:16:57.190514 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
I20260809 23:16:57.190738 26609 tablet_service.cc:1467] Processing CreateTablet for tablet 0e5dee0f49734355a94be236989b55e2 (DEFAULT_TABLE table=test-table [id=f5e08db25111413fa3033b91ba88351a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:16:57.190745 26184 tablet_service.cc:1467] Processing CreateTablet for tablet 0e5dee0f49734355a94be236989b55e2 (DEFAULT_TABLE table=test-table [id=f5e08db25111413fa3033b91ba88351a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:16:57.191103 26609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e5dee0f49734355a94be236989b55e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:57.191114 26184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e5dee0f49734355a94be236989b55e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:57.192723 26394 tablet_service.cc:1467] Processing CreateTablet for tablet 0e5dee0f49734355a94be236989b55e2 (DEFAULT_TABLE table=test-table [id=f5e08db25111413fa3033b91ba88351a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:16:57.193009 26394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e5dee0f49734355a94be236989b55e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:57.193647 26745 tablet_bootstrap.cc:492] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1: Bootstrap starting.
I20260809 23:16:57.194011 26748 tablet_bootstrap.cc:492] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: Bootstrap starting.
I20260809 23:16:57.194350 26745 tablet_bootstrap.cc:654] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:57.194660 26748 tablet_bootstrap.cc:654] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:57.194823 26745 log.cc:824] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:57.195180 26748 log.cc:824] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:57.195677 26745 tablet_bootstrap.cc:492] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1: No bootstrap required, opened a new log
I20260809 23:16:57.195796 26745 ts_tablet_manager.cc:1397] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:16:57.195923 26748 tablet_bootstrap.cc:492] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: No bootstrap required, opened a new log
I20260809 23:16:57.195990 26748 ts_tablet_manager.cc:1397] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:16:57.196240 26754 tablet_bootstrap.cc:492] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e: Bootstrap starting.
I20260809 23:16:57.196821 26754 tablet_bootstrap.cc:654] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:57.197268 26754 log.cc:824] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:57.197649 26745 raft_consensus.cc:348] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } } peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } }
I20260809 23:16:57.197809 26745 raft_consensus.cc:374] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:57.197840 26745 raft_consensus.cc:729] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7f70a187e174dd28553c73cccc1b7d1, State: Initialized, Role: FOLLOWER
I20260809 23:16:57.197777 26748 raft_consensus.cc:348] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } } peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } }
I20260809 23:16:57.197924 26754 tablet_bootstrap.cc:492] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e: No bootstrap required, opened a new log
I20260809 23:16:57.197921 26748 raft_consensus.cc:374] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:57.197955 26748 raft_consensus.cc:729] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2eb4bd3849c484b9b62c1740463113d, State: Initialized, Role: FOLLOWER
I20260809 23:16:57.197978 26754 ts_tablet_manager.cc:1397] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:16:57.197976 26745 consensus_queue.cc:260] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } } peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } }
I20260809 23:16:57.198037 26748 consensus_queue.cc:260] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } } peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } }
I20260809 23:16:57.198251 26748 ts_tablet_manager.cc:1428] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:16:57.198253 26721 heartbeater.cc:503] Master 127.2.76.254:45069 was elected leader, sending a full tablet report...
I20260809 23:16:57.198241 26745 ts_tablet_manager.cc:1428] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:16:57.198506 26287 heartbeater.cc:503] Master 127.2.76.254:45069 was elected leader, sending a full tablet report...
I20260809 23:16:57.199702 26754 raft_consensus.cc:348] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } } peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } }
I20260809 23:16:57.199826 26754 raft_consensus.cc:374] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:57.199844 26754 raft_consensus.cc:729] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939dac5520824a25bbb09b7375f14c6e, State: Initialized, Role: FOLLOWER
I20260809 23:16:57.199935 26754 consensus_queue.cc:260] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } } peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } }
I20260809 23:16:57.200205 26498 heartbeater.cc:503] Master 127.2.76.254:45069 was elected leader, sending a full tablet report...
I20260809 23:16:57.200206 26754 ts_tablet_manager.cc:1428] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260809 23:16:57.229880 26499 tablet.cc:2369] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:16:57.340510 26288 tablet.cc:2369] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:16:57.405907 26722 tablet.cc:2369] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:16:57.492123 26756 raft_consensus.cc:482] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:16:57.492285 26756 raft_consensus.cc:504] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } } peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } }
I20260809 23:16:57.492648 26756 leader_election.cc:288] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2eb4bd3849c484b9b62c1740463113d (127.2.76.193:44045), 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437)
I20260809 23:16:57.496052 26743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.492920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:56161 (local address 127.2.76.193:44045)
0809 23:16:57.492959 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:57.492961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:57.493130 (+ 169us) server_negotiation.cc:407] Connection header received
0809 23:16:57.493164 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:57.493165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:57.493175 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:57.493231 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:57.493362 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.493942 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.495208 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.495702 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.495776 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.495784 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:57.495952 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 23:16:57.495989 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:16:57.496074 26724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.492828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:36437 (local address 127.2.76.195:56643)
0809 23:16:57.492885 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:57.492898 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:57.492949 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:57.493092 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:57.493094 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:57.493095 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:57.493167 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.493168 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.493987 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.493988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.495119 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.495122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.495783 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.495784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.495871 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.495874 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:16:57.495928 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 23:16:57.495981 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":63}
I20260809 23:16:57.496081 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.492828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:44045 (local address 127.2.76.195:56161)
0809 23:16:57.493040 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:57.493047 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:57.493109 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:57.493242 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:57.493242 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:57.493244 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:57.493300 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.493301 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.494016 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.494016 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.495119 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.495122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.495785 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.495786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.495869 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.495874 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:16:57.495928 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 23:16:57.495980 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":63,"thread_start_us":88,"threads_started":1}
I20260809 23:16:57.496050 26742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.492916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:56643 (local address 127.2.76.194:36437)
0809 23:16:57.492956 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:57.492957 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:57.492987 (+ 30us) server_negotiation.cc:407] Connection header received
0809 23:16:57.493014 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:57.493014 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:57.493031 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:57.493067 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:57.493235 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.493900 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.495206 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.495715 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.495776 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.495784 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:57.495952 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 23:16:57.495985 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:16:57.496567 26428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5dee0f49734355a94be236989b55e2" candidate_uuid: "b7f70a187e174dd28553c73cccc1b7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939dac5520824a25bbb09b7375f14c6e" is_pre_election: true
I20260809 23:16:57.496579 26212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5dee0f49734355a94be236989b55e2" candidate_uuid: "b7f70a187e174dd28553c73cccc1b7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2eb4bd3849c484b9b62c1740463113d" is_pre_election: true
I20260809 23:16:57.496717 26428 raft_consensus.cc:2454] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7f70a187e174dd28553c73cccc1b7d1 in term 0.
I20260809 23:16:57.496739 26212 raft_consensus.cc:2454] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7f70a187e174dd28553c73cccc1b7d1 in term 0.
I20260809 23:16:57.496963 26533 leader_election.cc:302] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2eb4bd3849c484b9b62c1740463113d, b7f70a187e174dd28553c73cccc1b7d1; no voters:
I20260809 23:16:57.497099 26756 raft_consensus.cc:2781] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:16:57.497152 26756 raft_consensus.cc:482] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:16:57.497174 26756 raft_consensus.cc:3037] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:57.497835 26756 raft_consensus.cc:504] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } } peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } }
I20260809 23:16:57.497963 26756 leader_election.cc:288] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2eb4bd3849c484b9b62c1740463113d (127.2.76.193:44045), 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437)
I20260809 23:16:57.498147 26212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5dee0f49734355a94be236989b55e2" candidate_uuid: "b7f70a187e174dd28553c73cccc1b7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2eb4bd3849c484b9b62c1740463113d"
I20260809 23:16:57.498153 26428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e5dee0f49734355a94be236989b55e2" candidate_uuid: "b7f70a187e174dd28553c73cccc1b7d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939dac5520824a25bbb09b7375f14c6e"
I20260809 23:16:57.498225 26428 raft_consensus.cc:3037] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:57.498225 26212 raft_consensus.cc:3037] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:57.499112 26428 raft_consensus.cc:2454] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7f70a187e174dd28553c73cccc1b7d1 in term 1.
I20260809 23:16:57.499112 26212 raft_consensus.cc:2454] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7f70a187e174dd28553c73cccc1b7d1 in term 1.
I20260809 23:16:57.499296 26533 leader_election.cc:302] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2eb4bd3849c484b9b62c1740463113d, b7f70a187e174dd28553c73cccc1b7d1; no voters:
I20260809 23:16:57.499390 26756 raft_consensus.cc:2781] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:16:57.499572 26756 raft_consensus.cc:686] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [term 1 LEADER]: Becoming Leader. State: Replica: b7f70a187e174dd28553c73cccc1b7d1, State: Running, Role: LEADER
I20260809 23:16:57.499667 26756 consensus_queue.cc:237] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } } peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } }
I20260809 23:16:57.500531 25986 catalog_manager.cc:5495] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195). New cstate: current_term: 1 leader_uuid: "b7f70a187e174dd28553c73cccc1b7d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } health_report { overall_health: UNKNOWN } } }
I20260809 23:16:57.540450 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.538161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56838 (local address 127.2.76.195:36841)
0809 23:16:57.538225 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:57.538228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:57.538237 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:16:57.538304 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:57.538304 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:57.538320 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:57.538399 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:16:57.538527 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.538929 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.539624 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.539814 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.539901 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.540115 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:16:57.540206 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:16:57.540301 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:57.540325 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 23:16:57.540346 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 23:16:57.542124 26428 raft_consensus.cc:1261] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e [term 1 FOLLOWER]: Refusing update from remote peer b7f70a187e174dd28553c73cccc1b7d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:16:57.542131 26212 raft_consensus.cc:1261] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Refusing update from remote peer b7f70a187e174dd28553c73cccc1b7d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:16:57.542490 26766 consensus_queue.cc:1035] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:16:57.542614 26756 consensus_queue.cc:1035] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:16:57.545583 26774 mvcc.cc:205] Tried to move back new op lower bound from 7316756142250049536 to 7316756142079250432. Current Snapshot: MvccSnapshot[applied={T|T < 7316756142250049536 or (T in {7316756142250049536})}]
I20260809 23:16:57.546629 25987 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:16:57.546751 25987 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:16:57.547506 25987 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: Sending DeleteTablet for 3 replicas of tablet 0e5dee0f49734355a94be236989b55e2
I20260809 23:16:57.547833 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 26088
I20260809 23:16:57.547817 26184 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e5dee0f49734355a94be236989b55e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:16:57 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:43640
I20260809 23:16:57.547856 26394 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e5dee0f49734355a94be236989b55e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:16:57 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:48806
I20260809 23:16:57.551200 26609 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e5dee0f49734355a94be236989b55e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:16:57 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:56824
I20260809 23:16:57.551487 26787 tablet_replica.cc:330] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1: stopping tablet replica
I20260809 23:16:57.551596 26787 raft_consensus.cc:2229] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:16:57.551748 26787 raft_consensus.cc:2258] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:16:57.551927 26786 tablet_replica.cc:330] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e: stopping tablet replica
I20260809 23:16:57.552007 26786 raft_consensus.cc:2229] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:16:57.552099 26786 raft_consensus.cc:2258] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:16:57.554144 26786 ts_tablet_manager.cc:1905] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:16:57.555933 26786 ts_tablet_manager.cc:1918] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:16:57.556016 26786 log.cc:1196] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/0e5dee0f49734355a94be236989b55e2
I20260809 23:16:57.556241 26786 ts_tablet_manager.cc:1939] T 0e5dee0f49734355a94be236989b55e2 P 939dac5520824a25bbb09b7375f14c6e: Deleting consensus metadata
I20260809 23:16:57.557008 25964 catalog_manager.cc:4841] TS 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437): tablet 0e5dee0f49734355a94be236989b55e2 (table test-table [id=f5e08db25111413fa3033b91ba88351a]) successfully deleted
I20260809 23:16:57.569554 26787 ts_tablet_manager.cc:1905] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:16:57.571090 26787 ts_tablet_manager.cc:1918] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:16:57.571166 26787 log.cc:1196] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/0e5dee0f49734355a94be236989b55e2
I20260809 23:16:57.571439 26787 ts_tablet_manager.cc:1939] T 0e5dee0f49734355a94be236989b55e2 P b7f70a187e174dd28553c73cccc1b7d1: Deleting consensus metadata
I20260809 23:16:57.571851 25958 catalog_manager.cc:4841] TS b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841): tablet 0e5dee0f49734355a94be236989b55e2 (table test-table [id=f5e08db25111413fa3033b91ba88351a]) successfully deleted
W20260809 23:16:57.574416 25962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:44045 (error 108)
I20260809 23:16:57.574678 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 26299
I20260809 23:16:57.579859 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.574850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:44045 (local address 127.0.0.1:43646)
0809 23:16:57.579753 (+ 4903us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:57.579807 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:44045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4879}
W20260809 23:16:57.579999 25962 catalog_manager.cc:4584] TS a2eb4bd3849c484b9b62c1740463113d (127.2.76.193:44045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e5dee0f49734355a94be236989b55e2: Network error: Client connection negotiation failed: client connection to 127.2.76.193:44045: connect: Connection refused (error 111)
I20260809 23:16:57.590252 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 26508
I20260809 23:16:57.603139 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 25936
I20260809 23:16:57.614441 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45069
--webserver_interface=127.2.76.254
--webserver_port=41625
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:45069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:57.688709 26790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:57.688868 26790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:57.688887 26790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:57.689826 26790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:16:57.689857 26790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:57.689870 26790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:57.689882 26790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:16:57.689893 26790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:16:57.691187 26790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:45069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=41625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:57.691350 26790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:57.691488 26790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:57.699486 26804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:57.699494 26802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:57.699503 26801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:57.699693 26790 server_base.cc:1034] running on GCE node
I20260809 23:16:57.699956 26790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:57.700292 26790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:57.701443 26790 hybrid_clock.cc:648] HybridClock initialized: now 1786317417701422 us; error 32 us; skew 500 ppm
Aug 09 23:16:57 dist-test-slave-7vjv krb5kdc[25913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317417, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:57.703778 26790 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:16:57.704264 26790 webserver.cc:460] Webserver started at http://127.2.76.254:41625/ using document root <none> and password file <none>
I20260809 23:16:57.704488 26790 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:57.704566 26790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:57.705752 26790 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:16:57.706321 26815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:57.706499 26790 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:16:57.706593 26790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "ed5802d1b2df47ceb62dd6d94e2283cc"
format_stamp: "Formatted at 2026-08-09 23:16:56 on dist-test-slave-7vjv"
server_key: "cc52728dc0a64143798ff41d2c4040ad"
server_key_iv: "d6d9774defa2c8f1738fdcfeac375ef9"
server_key_version: "encryptionkey@0"
I20260809 23:16:57.706923 26790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:57.731482 26790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:57.733644 26790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:57.733803 26790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:57.737650 26790 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:45069
I20260809 23:16:57.737680 26911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:45069 every 8 connection(s)
I20260809 23:16:57.738018 26790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:16:57.739380 26912 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:16:57.739764 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 26790
I20260809 23:16:57.740058 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:44045
--local_ip_for_outbound_sockets=127.2.76.193
--tserver_master_addrs=127.2.76.254:45069
--webserver_port=36953
--webserver_interface=127.2.76.193
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:16:57.740171 26912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: Bootstrap starting.
I20260809 23:16:57.749099 26912 log.cc:824] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:57.750675 26912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:16:57.750949 26912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: Bootstrap complete.
I20260809 23:16:57.753085 26912 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } }
I20260809 23:16:57.753279 26912 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed5802d1b2df47ceb62dd6d94e2283cc, State: Initialized, Role: FOLLOWER
I20260809 23:16:57.753425 26912 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } }
I20260809 23:16:57.753501 26912 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:16:57.753553 26912 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:16:57.753599 26912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 1 FOLLOWER]: Advancing to term 2
I20260809 23:16:57.754484 26912 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } }
I20260809 23:16:57.754585 26912 leader_election.cc:302] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed5802d1b2df47ceb62dd6d94e2283cc; no voters:
I20260809 23:16:57.754767 26912 leader_election.cc:288] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 23:16:57.754799 26918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 2 FOLLOWER]: Leader election won for term 2
I20260809 23:16:57.754933 26918 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 2 LEADER]: Becoming Leader. State: Replica: ed5802d1b2df47ceb62dd6d94e2283cc, State: Running, Role: LEADER
I20260809 23:16:57.755007 26912 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:16:57.755077 26918 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } }
I20260809 23:16:57.755313 26918 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed5802d1b2df47ceb62dd6d94e2283cc. Latest consensus state: current_term: 2 leader_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } } }
I20260809 23:16:57.755412 26918 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:57.755316 26919 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } } }
I20260809 23:16:57.755645 26919 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:57.755743 26926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:16:57.756678 26926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:16:57.756775 26926 catalog_manager.cc:1259] Loaded cluster ID: 78a50c2b4d754a8fa383180f92ab164e
I20260809 23:16:57.756819 26926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:16:57.757193 26926 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 23:16:57.761220 26926 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 23:16:57.763916 26926 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:16:57.764070 26926 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: Loaded TSK: 0
I20260809 23:16:57.764281 26926 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 23:16:57.819176 26914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:57.819408 26914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:57.819445 26914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:57.821291 26914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:57.821344 26914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:57.821398 26914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:16:57.823837 26914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:44045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=36953
--tserver_master_addrs=127.2.76.254:45069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:57.824674 26914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:57.824920 26914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:57.829593 26951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:57.829703 26914 server_base.cc:1034] running on GCE node
W20260809 23:16:57.829566 26954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:57.829576 26957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:57.830061 26914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:57.830293 26914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:57.831451 26914 hybrid_clock.cc:648] HybridClock initialized: now 1786317417831439 us; error 29 us; skew 500 ppm
Aug 09 23:16:57 dist-test-slave-7vjv krb5kdc[25913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317417, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:57.833900 26914 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:16:57.834344 26914 webserver.cc:460] Webserver started at http://127.2.76.193:36953/ using document root <none> and password file <none>
I20260809 23:16:57.834553 26914 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:57.834635 26914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:57.835857 26914 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:16:57.836602 26968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:57.836766 26914 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:57.836858 26914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "a2eb4bd3849c484b9b62c1740463113d"
format_stamp: "Formatted at 2026-08-09 23:16:56 on dist-test-slave-7vjv"
server_key: "5c4171e409a075bf70b6580b93c95f90"
server_key_iv: "528ec0dc047ad3964809d542ccec23b7"
server_key_version: "encryptionkey@0"
I20260809 23:16:57.837137 26914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:57.859012 26914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:57.860884 26914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:57.861016 26914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:57.861229 26914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:57.861667 26983 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 23:16:57.862848 26914 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 23:16:57.862893 26914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260809 23:16:57.862922 26914 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 23:16:57.863480 26914 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 23:16:57.863523 26914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 23:16:57.863565 26983 tablet_bootstrap.cc:492] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: Bootstrap starting.
I20260809 23:16:57.870639 26914 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:44045
I20260809 23:16:57.870785 27162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:44045 every 8 connection(s)
I20260809 23:16:57.871109 26914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:16:57.874850 26983 log.cc:824] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:57.875327 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 26914
I20260809 23:16:57.875623 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:36437
--local_ip_for_outbound_sockets=127.2.76.194
--tserver_master_addrs=127.2.76.254:45069
--webserver_port=32929
--webserver_interface=127.2.76.194
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:16:57.877349 26983 tablet_bootstrap.cc:492] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:16:57.877632 26983 tablet_bootstrap.cc:492] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: Bootstrap complete.
I20260809 23:16:57.878836 26983 ts_tablet_manager.cc:1397] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
Aug 09 23:16:57 dist-test-slave-7vjv krb5kdc[25913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317417, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:57.880364 26983 raft_consensus.cc:348] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } } peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } }
I20260809 23:16:57.880723 26983 raft_consensus.cc:729] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2eb4bd3849c484b9b62c1740463113d, State: Initialized, Role: FOLLOWER
I20260809 23:16:57.880867 26983 consensus_queue.cc:260] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } } peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } }
I20260809 23:16:57.881166 26983 ts_tablet_manager.cc:1428] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260809 23:16:57.883419 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.871784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:33817)
0809 23:16:57.871989 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:57.872060 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:57.872542 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:57.873012 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:57.873015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:57.873303 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:57.873538 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.873543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.875080 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.875081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.877546 (+ 2465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.877549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.878153 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.878154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.878270 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.878769 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:57.878777 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:57.880189 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:57.882019 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:57.882026 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:57.882027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:57.882228 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:57.882464 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:57.882465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:57.882465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:57.882570 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:57.882897 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:57.882904 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:57.883018 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 23:16:57.883102 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 23:16:57.883249 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":106,"threads_started":1}
I20260809 23:16:57.883419 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.871894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:33817 (local address 127.2.76.254:45069)
0809 23:16:57.872206 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:57.872218 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:57.872263 (+ 45us) server_negotiation.cc:407] Connection header received
0809 23:16:57.872765 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:57.872772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:57.872829 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:57.872966 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:57.873662 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.874954 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.877636 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.878043 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.878159 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.880283 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:57.880286 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:57.880287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:57.880298 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:57.881914 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:57.882308 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:57.882311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:57.882311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:57.882394 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:57.882632 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:57.882633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:57.882634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:57.882823 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:57.882904 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:57.883164 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 23:16:57.883270 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":81,"threads_started":1}
I20260809 23:16:57.884307 27163 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45069
I20260809 23:16:57.884388 27163 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:57.884567 27163 heartbeater.cc:511] Master 127.2.76.254:45069 requested a full tablet report, sending...
I20260809 23:16:57.885181 26846 ts_manager.cc:194] Registered new tserver with Master: a2eb4bd3849c484b9b62c1740463113d (127.2.76.193:44045)
I20260809 23:16:57.886118 26846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:33817
I20260809 23:16:57.887005 27163 heartbeater.cc:503] Master 127.2.76.254:45069 was elected leader, sending a full tablet report...
Aug 09 23:16:57 dist-test-slave-7vjv krb5kdc[25913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317417, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for kudu/127.2.76.193@KRBTEST.COM
I20260809 23:16:57.893121 27177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.885786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43648 (local address 127.2.76.193:44045)
0809 23:16:57.885903 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:57.885907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:57.886065 (+ 158us) server_negotiation.cc:407] Connection header received
0809 23:16:57.886125 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:57.886127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:57.886144 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:57.886211 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:57.886828 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.887699 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.888645 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.888869 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.888918 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.890728 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:57.890735 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:57.890735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:57.890741 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:57.892016 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:57.892373 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:57.892376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:57.892376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:57.892429 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:57.892622 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:57.892623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:57.892623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:57.892735 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:57.892815 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:57.892968 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 23:16:57.893050 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
I20260809 23:16:57.893121 27174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.885727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:44045 (local address 127.0.0.1:43648)
0809 23:16:57.885958 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:57.885988 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:57.886055 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:57.886276 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:57.886282 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:57.886688 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:57.886742 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.886745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.887778 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.887779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.888579 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.888581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.888935 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.888936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.889010 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.889363 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:57.889367 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:57.890631 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:57.892115 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:57.892121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:57.892123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:57.892298 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:57.892504 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:57.892504 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:57.892505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:57.892561 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:57.892800 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:57.892806 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:57.892850 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 23:16:57.892935 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 23:16:57.893039 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":71,"threads_started":1}
I20260809 23:16:57.893447 27052 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e5dee0f49734355a94be236989b55e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:16:57 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:43648
I20260809 23:16:57.893827 27178 tablet_replica.cc:330] stopping tablet replica
I20260809 23:16:57.893916 27178 raft_consensus.cc:2229] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:16:57.893968 27178 raft_consensus.cc:2258] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:16:57.894207 27178 ts_tablet_manager.cc:1905] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:16:57.896225 27178 ts_tablet_manager.cc:1918] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:16:57.896298 27178 log.cc:1196] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal/wals/0e5dee0f49734355a94be236989b55e2
I20260809 23:16:57.896509 27178 ts_tablet_manager.cc:1939] T 0e5dee0f49734355a94be236989b55e2 P a2eb4bd3849c484b9b62c1740463113d: Deleting consensus metadata
I20260809 23:16:57.896976 26820 catalog_manager.cc:4841] TS a2eb4bd3849c484b9b62c1740463113d (127.2.76.193:44045): tablet 0e5dee0f49734355a94be236989b55e2 (table test-table [id=f5e08db25111413fa3033b91ba88351a]) successfully deleted
W20260809 23:16:57.951334 27172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:57.951531 27172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:57.951586 27172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:57.952582 27172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:57.952639 27172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:57.952679 27172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:16:57.953919 27172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:36437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=32929
--tserver_master_addrs=127.2.76.254:45069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:57.954142 27172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:57.954339 27172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:57.958722 27195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:57.958714 27193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:57.958739 27197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:57.958990 27172 server_base.cc:1034] running on GCE node
I20260809 23:16:57.959168 27172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:57.959347 27172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:57.960484 27172 hybrid_clock.cc:648] HybridClock initialized: now 1786317417960474 us; error 24 us; skew 500 ppm
Aug 09 23:16:57 dist-test-slave-7vjv krb5kdc[25913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317417, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:57.962504 27172 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:16:57.962867 27172 webserver.cc:460] Webserver started at http://127.2.76.194:32929/ using document root <none> and password file <none>
I20260809 23:16:57.963079 27172 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:57.963150 27172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:57.964242 27172 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:16:57.964747 27211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:57.964895 27172 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:57.964998 27172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "939dac5520824a25bbb09b7375f14c6e"
format_stamp: "Formatted at 2026-08-09 23:16:56 on dist-test-slave-7vjv"
server_key: "1381b3c82653cdf86eab147bd59087f5"
server_key_iv: "0828a7865feb6f6a337551ecb12c289b"
server_key_version: "encryptionkey@0"
I20260809 23:16:57.965269 27172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:57.980618 27172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:57.982250 27172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:57.982386 27172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:57.982571 27172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:57.982822 27172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:57.982867 27172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:57.982893 27172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:57.982916 27172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:57.988337 27172 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:36437
I20260809 23:16:57.988394 27400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:36437 every 8 connection(s)
I20260809 23:16:57.988648 27172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:16:57.991616 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 27172
I20260809 23:16:57.991871 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:36841
--local_ip_for_outbound_sockets=127.2.76.195
--tserver_master_addrs=127.2.76.254:45069
--webserver_port=33771
--webserver_interface=127.2.76.195
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:16:57 dist-test-slave-7vjv krb5kdc[25913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317417, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:57.996857 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.989190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:43337 (local address 127.2.76.254:45069)
0809 23:16:57.989244 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:57.989246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:57.989421 (+ 175us) server_negotiation.cc:407] Connection header received
0809 23:16:57.989781 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:57.989782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:57.989793 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:57.989838 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:57.990547 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.990956 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.991751 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.992047 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.992140 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.994152 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:57.994153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:57.994154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:57.994160 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:57.995463 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:57.995941 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:57.995942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:57.995943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:57.995988 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:57.996265 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:57.996266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:57.996266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:57.996371 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:57.996442 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:57.996689 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 23:16:57.996773 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 23:16:57.996989 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:57.989203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:43337)
0809 23:16:57.989338 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:57.989365 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:57.989711 (+ 346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:57.989879 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:57.989881 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:57.990331 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:57.990475 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.990478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.991028 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.991028 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.991665 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:57.991667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:57.992149 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:57.992150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:57.992249 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:57.992651 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:57.992657 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:57.994057 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:57.995546 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:57.995554 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:57.995556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:57.995860 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:57.996067 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:57.996068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:57.996068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:57.996197 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:57.996436 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:57.996443 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:57.996571 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 23:16:57.996667 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 23:16:57.996800 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260809 23:16:57.997362 27402 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45069
I20260809 23:16:57.997462 27402 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:57.997643 27402 heartbeater.cc:511] Master 127.2.76.254:45069 requested a full tablet report, sending...
I20260809 23:16:57.998014 26846 ts_manager.cc:194] Registered new tserver with Master: 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437)
I20260809 23:16:57.998343 26846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:43337
W20260809 23:16:58.071118 27406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:58.071290 27406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:58.071312 27406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:58.072521 27406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:58.072563 27406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:58.072624 27406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:16:58.074011 27406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:36841
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=33771
--tserver_master_addrs=127.2.76.254:45069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:58.074229 27406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:58.074445 27406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:58.079228 27412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:58.079229 27413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:58.079243 27406 server_base.cc:1034] running on GCE node
W20260809 23:16:58.079216 27416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:58.079582 27406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:58.079808 27406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:58.080992 27406 hybrid_clock.cc:648] HybridClock initialized: now 1786317418080976 us; error 34 us; skew 500 ppm
Aug 09 23:16:58 dist-test-slave-7vjv krb5kdc[25913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317418, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:58.083282 27406 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:16:58.083694 27406 webserver.cc:460] Webserver started at http://127.2.76.195:33771/ using document root <none> and password file <none>
I20260809 23:16:58.083941 27406 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:58.083997 27406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:58.085290 27406 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:16:58.085799 27437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:58.085955 27406 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:16:58.086036 27406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "b7f70a187e174dd28553c73cccc1b7d1"
format_stamp: "Formatted at 2026-08-09 23:16:57 on dist-test-slave-7vjv"
server_key: "ec208ad32806b249ad5d693c0a313649"
server_key_iv: "c25f2c01b69e70eefe5785f95fbf345e"
server_key_version: "encryptionkey@0"
I20260809 23:16:58.086284 27406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:58.098536 27406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:58.100061 27406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:58.100214 27406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:58.100374 27406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:58.100575 27406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:58.100602 27406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:58.100620 27406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:58.100631 27406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:58.105872 27406 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:36841
I20260809 23:16:58.105949 27612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:36841 every 8 connection(s)
I20260809 23:16:58.106253 27406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:16:58.107726 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 27406
Aug 09 23:16:58 dist-test-slave-7vjv krb5kdc[25913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317418, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:16:58.114603 27623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.108058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54024 (local address 127.2.76.254:45069)
0809 23:16:58.108177 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:58.108179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:58.108219 (+ 40us) server_negotiation.cc:407] Connection header received
0809 23:16:58.108287 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:58.108288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:58.108314 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:58.108386 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:58.109106 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.109808 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.110487 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.110752 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.110800 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:58.111909 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:58.111910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:58.111910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:58.111918 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:58.113436 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:58.113785 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:58.113786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:58.113787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:58.113863 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:58.114066 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:58.114067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:58.114067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:58.114178 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:58.114383 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:58.114422 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 23:16:58.114500 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":81,"threads_started":1}
I20260809 23:16:58.115908 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.106889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:45099 (local address 127.2.76.254:45069)
0809 23:16:58.106931 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:58.106933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:58.107094 (+ 161us) server_negotiation.cc:407] Connection header received
0809 23:16:58.107486 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:58.107487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:58.107496 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:58.107543 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:16:58.108617 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.109296 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.110593 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.110769 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.110852 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:58.113006 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:16:58.113007 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:16:58.113008 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:16:58.113014 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:16:58.114422 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:58.115095 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:58.115095 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:58.115096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:58.115143 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:16:58.115400 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:16:58.115400 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:16:58.115401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:16:58.115539 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:16:58.115592 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:58.115804 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 23:16:58.115858 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:16:58.116165 27619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.106862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:45099)
0809 23:16:58.106998 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:58.107032 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:58.107416 (+ 384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:58.107584 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:58.107586 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:58.108285 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:16:58.108464 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:58.108469 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.109376 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.109377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:58.110514 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:58.110517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.110880 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.110881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:58.110985 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:58.111485 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:16:58.111492 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:16:58.112913 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:16:58.114522 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:58.114528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:58.114530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:58.114933 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:58.115219 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:16:58.115220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:16:58.115220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:16:58.115326 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:16:58.115599 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:16:58.115604 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:16:58.115712 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 23:16:58.115790 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 23:16:58.115888 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260809 23:16:58.116598 27613 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45069
I20260809 23:16:58.116700 27613 heartbeater.cc:464] Registering TS with master...
I20260809 23:16:58.116870 27613 heartbeater.cc:511] Master 127.2.76.254:45069 requested a full tablet report, sending...
I20260809 23:16:58.117252 26846 ts_manager.cc:194] Registered new tserver with Master: b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841)
I20260809 23:16:58.117609 26846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:45099
I20260809 23:16:58.126121 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:16:58.128592 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.126415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54028 (local address 127.2.76.254:45069)
0809 23:16:58.126472 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:58.126474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:58.126567 (+ 93us) server_negotiation.cc:407] Connection header received
0809 23:16:58.126636 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:58.126636 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:58.126672 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:58.126739 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:16:58.126833 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.127223 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.127885 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.128057 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.128153 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:58.128313 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:16:58.128396 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:16:58.128478 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:58.128512 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 23:16:58.128527 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:16:58.128827 26846 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:54028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:16:58.130175 26846 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:16:58.132033 27052 tablet_service.cc:1467] Processing CreateTablet for tablet 6093ce1bbd29464b8ecaf77ac4323399 (DEFAULT_TABLE table=test-table [id=61a08371d09b44ceb6724bef12e6e97b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:16:58.132253 27052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6093ce1bbd29464b8ecaf77ac4323399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:58.133730 26983 tablet_bootstrap.cc:492] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d: Bootstrap starting.
I20260809 23:16:58.134022 26983 tablet_bootstrap.cc:654] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:58.134831 26983 tablet_bootstrap.cc:492] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d: No bootstrap required, opened a new log
I20260809 23:16:58.134886 26983 ts_tablet_manager.cc:1397] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:16:58.135051 26983 raft_consensus.cc:348] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:16:58.135126 26983 raft_consensus.cc:374] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:58.135175 26983 raft_consensus.cc:729] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2eb4bd3849c484b9b62c1740463113d, State: Initialized, Role: FOLLOWER
I20260809 23:16:58.135216 26983 consensus_queue.cc:260] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:16:58.135298 26983 ts_tablet_manager.cc:1428] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:58.136016 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.131708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:36841 (local address 127.0.0.1:56848)
0809 23:16:58.131823 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:58.131830 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:58.131863 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:58.132311 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:58.132312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:58.132315 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:58.132350 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:58.132352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.133773 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.133774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:58.134830 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:58.134833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.135748 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.135749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:58.135809 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:58.135812 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:16:58.135861 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 23:16:58.135903 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":82,"threads_started":1}
I20260809 23:16:58.135999 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.131965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56848 (local address 127.2.76.195:36841)
0809 23:16:58.132134 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:58.132139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:58.132149 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:16:58.132182 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:58.132183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:58.132198 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:58.132293 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:58.132454 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.133662 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.134925 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.135544 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.135611 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:58.135677 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:58.135878 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 23:16:58.135909 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260809 23:16:58.136080 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.131772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48812 (local address 127.2.76.194:36437)
0809 23:16:58.132226 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:58.132230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:58.132239 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:16:58.132275 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:58.132277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:58.132303 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:58.132510 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:58.132610 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.133677 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.134957 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.135622 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.135692 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:58.135748 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:58.135915 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 23:16:58.135982 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":91,"threads_started":1}
I20260809 23:16:58.136027 27174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.131651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:36437 (local address 127.0.0.1:48812)
0809 23:16:58.131683 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:58.131693 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:58.131728 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:58.132508 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:58.132508 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:58.132510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:58.132551 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:58.132552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.133842 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.133842 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:58.134873 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:58.134876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.135699 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.135700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:58.135771 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:58.135795 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 23:16:58.135847 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 23:16:58.135894 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":61}
I20260809 23:16:58.136885 27294 tablet_service.cc:1467] Processing CreateTablet for tablet 6093ce1bbd29464b8ecaf77ac4323399 (DEFAULT_TABLE table=test-table [id=61a08371d09b44ceb6724bef12e6e97b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:16:58.136889 27503 tablet_service.cc:1467] Processing CreateTablet for tablet 6093ce1bbd29464b8ecaf77ac4323399 (DEFAULT_TABLE table=test-table [id=61a08371d09b44ceb6724bef12e6e97b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:16:58.137158 27503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6093ce1bbd29464b8ecaf77ac4323399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:58.137189 27294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6093ce1bbd29464b8ecaf77ac4323399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:16:58.139662 27634 tablet_bootstrap.cc:492] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e: Bootstrap starting.
I20260809 23:16:58.139536 27633 tablet_bootstrap.cc:492] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1: Bootstrap starting.
I20260809 23:16:58.140340 27634 tablet_bootstrap.cc:654] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:58.140358 27633 tablet_bootstrap.cc:654] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1: Neither blocks nor log segments found. Creating new log.
I20260809 23:16:58.140785 27634 log.cc:824] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:58.140805 27633 log.cc:824] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:58.141480 27634 tablet_bootstrap.cc:492] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e: No bootstrap required, opened a new log
I20260809 23:16:58.141494 27633 tablet_bootstrap.cc:492] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1: No bootstrap required, opened a new log
I20260809 23:16:58.141537 27634 ts_tablet_manager.cc:1397] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:16:58.141567 27633 ts_tablet_manager.cc:1397] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:16:58.142978 27633 raft_consensus.cc:348] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:16:58.143038 27634 raft_consensus.cc:348] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:16:58.143128 27633 raft_consensus.cc:374] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:58.143172 27633 raft_consensus.cc:729] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7f70a187e174dd28553c73cccc1b7d1, State: Initialized, Role: FOLLOWER
I20260809 23:16:58.143173 27634 raft_consensus.cc:374] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:16:58.143210 27634 raft_consensus.cc:729] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939dac5520824a25bbb09b7375f14c6e, State: Initialized, Role: FOLLOWER
I20260809 23:16:58.143282 27633 consensus_queue.cc:260] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:16:58.143326 27634 consensus_queue.cc:260] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:16:58.143633 27634 ts_tablet_manager.cc:1428] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:16:58.143627 27613 heartbeater.cc:503] Master 127.2.76.254:45069 was elected leader, sending a full tablet report...
I20260809 23:16:58.143656 27402 heartbeater.cc:503] Master 127.2.76.254:45069 was elected leader, sending a full tablet report...
I20260809 23:16:58.143628 27633 ts_tablet_manager.cc:1428] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 23:16:58.152217 27173 raft_consensus.cc:482] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:16:58.152323 27173 raft_consensus.cc:504] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:16:58.152609 27173 leader_election.cc:288] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437), b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841)
I20260809 23:16:58.155328 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.152837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:35451 (local address 127.2.76.194:36437)
0809 23:16:58.152874 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:58.152875 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:58.152882 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:16:58.152900 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:58.152901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:58.152907 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:58.152976 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:58.153122 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.153557 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.154561 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.154969 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.155042 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:58.155048 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:58.155218 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 23:16:58.155252 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:16:58.155416 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.152719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:36437 (local address 127.2.76.193:35451)
0809 23:16:58.152754 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:58.152766 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:58.152815 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:58.152979 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:58.152980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:58.152981 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:58.153046 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:58.153048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.153644 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.153645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:58.154463 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:58.154466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.155042 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.155044 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:58.155112 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:58.155137 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 23:16:58.155205 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 23:16:58.155317 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":72}
I20260809 23:16:58.155437 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.152801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:59639 (local address 127.2.76.195:36841)
0809 23:16:58.152841 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:58.152843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:58.152978 (+ 135us) server_negotiation.cc:407] Connection header received
0809 23:16:58.153045 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:58.153045 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:58.153053 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:58.153105 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:16:58.153264 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.153695 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.154746 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.155056 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.155130 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:58.155137 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:58.155338 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 23:16:58.155368 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:16:58.155697 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.152739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:36841 (local address 127.2.76.193:59639)
0809 23:16:58.152902 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:58.152911 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:58.152991 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:58.153109 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:16:58.153110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:16:58.153111 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:16:58.153200 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:58.153201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.153780 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.153781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:58.154651 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:16:58.154653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.155152 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.155153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:16:58.155220 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:58.155268 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 23:16:58.155333 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 23:16:58.155371 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":10,"thread_start_us":80,"threads_started":1}
I20260809 23:16:58.155768 27331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6093ce1bbd29464b8ecaf77ac4323399" candidate_uuid: "a2eb4bd3849c484b9b62c1740463113d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939dac5520824a25bbb09b7375f14c6e" is_pre_election: true
I20260809 23:16:58.155874 27331 raft_consensus.cc:2454] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2eb4bd3849c484b9b62c1740463113d in term 0.
I20260809 23:16:58.156001 27537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6093ce1bbd29464b8ecaf77ac4323399" candidate_uuid: "a2eb4bd3849c484b9b62c1740463113d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7f70a187e174dd28553c73cccc1b7d1" is_pre_election: true
I20260809 23:16:58.156111 26977 leader_election.cc:302] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 939dac5520824a25bbb09b7375f14c6e, a2eb4bd3849c484b9b62c1740463113d; no voters:
I20260809 23:16:58.156159 27537 raft_consensus.cc:2454] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2eb4bd3849c484b9b62c1740463113d in term 0.
I20260809 23:16:58.156232 27173 raft_consensus.cc:2781] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:16:58.156296 27173 raft_consensus.cc:482] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:16:58.156322 27173 raft_consensus.cc:3037] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:58.156896 27173 raft_consensus.cc:504] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:16:58.157043 27173 leader_election.cc:288] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 1 election: Requested vote from peers 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437), b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841)
I20260809 23:16:58.157181 27331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6093ce1bbd29464b8ecaf77ac4323399" candidate_uuid: "a2eb4bd3849c484b9b62c1740463113d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939dac5520824a25bbb09b7375f14c6e"
I20260809 23:16:58.157196 27537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6093ce1bbd29464b8ecaf77ac4323399" candidate_uuid: "a2eb4bd3849c484b9b62c1740463113d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7f70a187e174dd28553c73cccc1b7d1"
I20260809 23:16:58.157269 27331 raft_consensus.cc:3037] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:58.157266 27537 raft_consensus.cc:3037] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:16:58.157860 27537 raft_consensus.cc:2454] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2eb4bd3849c484b9b62c1740463113d in term 1.
I20260809 23:16:58.157927 27331 raft_consensus.cc:2454] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2eb4bd3849c484b9b62c1740463113d in term 1.
I20260809 23:16:58.158041 26969 leader_election.cc:302] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2eb4bd3849c484b9b62c1740463113d, b7f70a187e174dd28553c73cccc1b7d1; no voters:
I20260809 23:16:58.158134 27173 raft_consensus.cc:2781] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:16:58.158281 27173 raft_consensus.cc:686] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 LEADER]: Becoming Leader. State: Replica: a2eb4bd3849c484b9b62c1740463113d, State: Running, Role: LEADER
I20260809 23:16:58.158352 27173 consensus_queue.cc:237] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:16:58.158981 26845 catalog_manager.cc:5495] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d reported cstate change: term changed from 0 to 1, leader changed from <none> to a2eb4bd3849c484b9b62c1740463113d (127.2.76.193). New cstate: current_term: 1 leader_uuid: "a2eb4bd3849c484b9b62c1740463113d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } health_report { overall_health: HEALTHY } } }
I20260809 23:16:58.166162 27177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.164366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43658 (local address 127.2.76.193:44045)
0809 23:16:58.164403 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:16:58.164405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:16:58.164413 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:16:58.164476 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:16:58.164477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:16:58.164490 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:16:58.164541 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:16:58.164625 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.165028 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.165619 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:16:58.165795 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:16:58.165839 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:16:58.165951 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:16:58.166025 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:16:58.166073 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 23:16:58.166109 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 23:16:58.166123 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:16:58.167368 27537 raft_consensus.cc:1261] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1 [term 1 FOLLOWER]: Refusing update from remote peer a2eb4bd3849c484b9b62c1740463113d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:16:58.167388 27331 raft_consensus.cc:1261] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e [term 1 FOLLOWER]: Refusing update from remote peer a2eb4bd3849c484b9b62c1740463113d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:16:58.167665 27173 consensus_queue.cc:1035] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [LEADER]: Connected to new peer: Peer: permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:16:58.167778 27647 consensus_queue.cc:1035] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:16:58.169914 27652 mvcc.cc:205] Tried to move back new op lower bound from 7316756144811438080 to 7316756144777097216. Current Snapshot: MvccSnapshot[applied={T|T < 7316756144811438080}]
I20260809 23:16:58.170642 27656 mvcc.cc:205] Tried to move back new op lower bound from 7316756144811438080 to 7316756144777097216. Current Snapshot: MvccSnapshot[applied={T|T < 7316756144811438080 or (T in {7316756144811438080})}]
I20260809 23:16:58.172433 26846 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:54028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:16:58.172499 26846 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:54028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:16:58.173141 26846 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: Sending DeleteTablet for 3 replicas of tablet 6093ce1bbd29464b8ecaf77ac4323399
I20260809 23:16:58.173408 27294 tablet_service.cc:1514] Processing DeleteTablet for tablet 6093ce1bbd29464b8ecaf77ac4323399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:16:58 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:48812
I20260809 23:16:58.173440 27503 tablet_service.cc:1514] Processing DeleteTablet for tablet 6093ce1bbd29464b8ecaf77ac4323399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:16:58 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:56848
I20260809 23:16:58.173620 27669 tablet_replica.cc:330] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1: stopping tablet replica
I20260809 23:16:58.173642 27052 tablet_service.cc:1514] Processing DeleteTablet for tablet 6093ce1bbd29464b8ecaf77ac4323399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:16:58 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:43648
I20260809 23:16:58.173709 27669 raft_consensus.cc:2229] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:16:58.173731 27178 tablet_replica.cc:330] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d: stopping tablet replica
I20260809 23:16:58.173776 27178 raft_consensus.cc:2229] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:16:58.173800 27669 raft_consensus.cc:2258] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:16:58.173841 27178 raft_consensus.cc:2258] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:16:58.173840 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 26914
I20260809 23:16:58.183947 27669 ts_tablet_manager.cc:1905] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:16:58.184954 27668 tablet_replica.cc:330] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e: stopping tablet replica
I20260809 23:16:58.185074 27668 raft_consensus.cc:2229] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:16:58.185200 27669 ts_tablet_manager.cc:1918] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:16:58.185230 27668 raft_consensus.cc:2258] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:16:58.185287 27669 log.cc:1196] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/6093ce1bbd29464b8ecaf77ac4323399
I20260809 23:16:58.185588 27669 ts_tablet_manager.cc:1939] T 6093ce1bbd29464b8ecaf77ac4323399 P b7f70a187e174dd28553c73cccc1b7d1: Deleting consensus metadata
I20260809 23:16:58.185659 27668 ts_tablet_manager.cc:1905] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:16:58.185982 26817 catalog_manager.cc:4841] TS b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841): tablet 6093ce1bbd29464b8ecaf77ac4323399 (table test-table [id=61a08371d09b44ceb6724bef12e6e97b]) successfully deleted
I20260809 23:16:58.187047 27668 ts_tablet_manager.cc:1918] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:16:58.187111 27668 log.cc:1196] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/6093ce1bbd29464b8ecaf77ac4323399
I20260809 23:16:58.187335 27668 ts_tablet_manager.cc:1939] T 6093ce1bbd29464b8ecaf77ac4323399 P 939dac5520824a25bbb09b7375f14c6e: Deleting consensus metadata
I20260809 23:16:58.187680 26822 catalog_manager.cc:4841] TS 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437): tablet 6093ce1bbd29464b8ecaf77ac4323399 (table test-table [id=61a08371d09b44ceb6724bef12e6e97b]) successfully deleted
W20260809 23:16:58.189512 26820 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:44045 (error 108)
I20260809 23:16:58.189829 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 27172
I20260809 23:16:58.190008 27174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.189885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:44045 (local address 127.0.0.1:43668)
0809 23:16:58.189932 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:58.189965 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:44045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 23:16:58.194821 26820 catalog_manager.cc:4584] TS a2eb4bd3849c484b9b62c1740463113d (127.2.76.193:44045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6093ce1bbd29464b8ecaf77ac4323399: Network error: Client connection negotiation failed: client connection to 127.2.76.193:44045: connect: Connection refused (error 111)
I20260809 23:16:58.209316 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 27406
I20260809 23:16:58.218972 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 26790
I20260809 23:16:58.227123 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45069
--webserver_interface=127.2.76.254
--webserver_port=41625
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:45069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:58.299432 27671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:58.299649 27671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:58.299683 27671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:58.300808 27671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:16:58.300868 27671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:58.300889 27671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:58.300907 27671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:16:58.300935 27671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:16:58.302251 27671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:45069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=41625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:58.302492 27671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:58.302680 27671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:16:58.307008 27671 server_base.cc:1034] running on GCE node
W20260809 23:16:58.306982 27685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:58.306982 27683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:58.306960 27682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:58.307329 27671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:58.307605 27671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:58.308743 27671 hybrid_clock.cc:648] HybridClock initialized: now 1786317418308732 us; error 27 us; skew 500 ppm
Aug 09 23:16:58 dist-test-slave-7vjv krb5kdc[25913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317418, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:58.310859 27671 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:16:58.311275 27671 webserver.cc:460] Webserver started at http://127.2.76.254:41625/ using document root <none> and password file <none>
I20260809 23:16:58.311446 27671 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:58.311481 27671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:58.312587 27671 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:16:58.313133 27694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:58.313297 27671 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:16:58.313381 27671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "ed5802d1b2df47ceb62dd6d94e2283cc"
format_stamp: "Formatted at 2026-08-09 23:16:56 on dist-test-slave-7vjv"
server_key: "cc52728dc0a64143798ff41d2c4040ad"
server_key_iv: "d6d9774defa2c8f1738fdcfeac375ef9"
server_key_version: "encryptionkey@0"
I20260809 23:16:58.313678 27671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:58.321913 27671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:58.323525 27671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:58.323678 27671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:58.327386 27671 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:45069
I20260809 23:16:58.327430 27792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:45069 every 8 connection(s)
I20260809 23:16:58.327749 27671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:16:58.329093 27793 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:16:58.329656 27793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: Bootstrap starting.
I20260809 23:16:58.331576 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 27671
I20260809 23:16:58.331836 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:44045
--local_ip_for_outbound_sockets=127.2.76.193
--tserver_master_addrs=127.2.76.254:45069
--webserver_port=36953
--webserver_interface=127.2.76.193
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:16:58.340416 27793 log.cc:824] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:58.343381 27793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:16:58.343683 27793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc: Bootstrap complete.
I20260809 23:16:58.346138 27793 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } }
I20260809 23:16:58.346349 27793 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed5802d1b2df47ceb62dd6d94e2283cc, State: Initialized, Role: FOLLOWER
I20260809 23:16:58.346517 27793 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } }
I20260809 23:16:58.346583 27793 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:16:58.346627 27793 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:16:58.346668 27793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 2 FOLLOWER]: Advancing to term 3
I20260809 23:16:58.347473 27793 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } }
I20260809 23:16:58.347582 27793 leader_election.cc:302] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed5802d1b2df47ceb62dd6d94e2283cc; no voters:
I20260809 23:16:58.347721 27793 leader_election.cc:288] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 23:16:58.347800 27799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 3 FOLLOWER]: Leader election won for term 3
I20260809 23:16:58.347949 27799 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [term 3 LEADER]: Becoming Leader. State: Replica: ed5802d1b2df47ceb62dd6d94e2283cc, State: Running, Role: LEADER
I20260809 23:16:58.347962 27793 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:16:58.348152 27799 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } }
I20260809 23:16:58.348489 27799 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed5802d1b2df47ceb62dd6d94e2283cc. Latest consensus state: current_term: 3 leader_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } } }
I20260809 23:16:58.348559 27803 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5802d1b2df47ceb62dd6d94e2283cc" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45069 } } }
I20260809 23:16:58.348639 27799 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:58.348646 27803 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed5802d1b2df47ceb62dd6d94e2283cc [sys.catalog]: This master's current role is: LEADER
I20260809 23:16:58.349063 27813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:16:58.349910 27813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:16:58.350011 27813 catalog_manager.cc:1259] Loaded cluster ID: 78a50c2b4d754a8fa383180f92ab164e
I20260809 23:16:58.350042 27813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 23:16:58.350324 27813 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe9733239ad google::LogMessage::Fail() at ??:0
@ 0x7fe97332688a google::LogMessage::SendToLog() at ??:0
@ 0x7fe9733234de google::LogMessage::Flush() at ??:0
@ 0x7fe973324a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe97b2440f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fe97b24d8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe9751f10d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe9751e6954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe97604a6db start_thread at ??:0
@ 0x7fe971e2161f clone at ??:0
W20260809 23:16:58.423071 27795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:58.423286 27795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:58.423331 27795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:58.424587 27795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:58.424660 27795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:58.424707 27795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:16:58.426082 27795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:44045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=36953
--tserver_master_addrs=127.2.76.254:45069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:58.426321 27795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:58.426517 27795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:58.431123 27856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:58.431159 27853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:58.431169 27854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:58.431444 27795 server_base.cc:1034] running on GCE node
I20260809 23:16:58.431676 27795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:58.431887 27795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:58.433043 27795 hybrid_clock.cc:648] HybridClock initialized: now 1786317418433032 us; error 30 us; skew 500 ppm
Aug 09 23:16:58 dist-test-slave-7vjv krb5kdc[25913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317418, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:58.435456 27795 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:16:58.435896 27795 webserver.cc:460] Webserver started at http://127.2.76.193:36953/ using document root <none> and password file <none>
I20260809 23:16:58.436156 27795 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:58.436226 27795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:58.437333 27795 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:16:58.437978 27866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:58.438191 27795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:16:58.438285 27795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "a2eb4bd3849c484b9b62c1740463113d"
format_stamp: "Formatted at 2026-08-09 23:16:56 on dist-test-slave-7vjv"
server_key: "5c4171e409a075bf70b6580b93c95f90"
server_key_iv: "528ec0dc047ad3964809d542ccec23b7"
server_key_version: "encryptionkey@0"
I20260809 23:16:58.438630 27795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:58.463752 27795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:58.465958 27795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:58.466096 27795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:58.466357 27795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:58.466794 27879 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 23:16:58.468142 27795 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 23:16:58.468235 27795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 23:16:58.468302 27795 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 23:16:58.469177 27795 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 23:16:58.469228 27795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 23:16:58.469316 27879 tablet_bootstrap.cc:492] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d: Bootstrap starting.
I20260809 23:16:58.477584 27795 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:44045
I20260809 23:16:58.478134 27795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:16:58.479755 27879 log.cc:824] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:16:58.480144 28061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:44045 every 8 connection(s)
I20260809 23:16:58.481845 27879 tablet_bootstrap.cc:492] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:16:58.482122 27879 tablet_bootstrap.cc:492] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d: Bootstrap complete.
I20260809 23:16:58.483085 27879 ts_tablet_manager.cc:1397] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.010s
I20260809 23:16:58.483942 27879 raft_consensus.cc:348] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:16:58.484196 27879 raft_consensus.cc:729] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2eb4bd3849c484b9b62c1740463113d, State: Initialized, Role: FOLLOWER
I20260809 23:16:58.484316 27879 consensus_queue.cc:260] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:16:58.484537 27879 ts_tablet_manager.cc:1428] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20260809 23:16:58.488243 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 27795
I20260809 23:16:58.488481 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:36437
--local_ip_for_outbound_sockets=127.2.76.194
--tserver_master_addrs=127.2.76.254:45069
--webserver_port=32929
--webserver_interface=127.2.76.194
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:58.566921 28075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:58.567112 28075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:58.567145 28075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:58.568193 28075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:58.568244 28075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:58.568282 28075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
W20260809 23:16:58.569216 28064 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:16:58.478861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:44507)
0809 23:16:58.479077 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:58.479129 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 23:16:58.479761 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:16:58.569039 (+ 89278us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":106,"threads_started":1}
W20260809 23:16:58.569440 28062 heartbeater.cc:644] Failed to heartbeat to 127.2.76.254:45069 (0 consecutive failures): Network error: Failed to ping master at 127.2.76.254:45069: Client connection negotiation failed: client connection to 127.2.76.254:45069: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 23:16:58.569762 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.569644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:44517)
0809 23:16:58.569696 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:58.569739 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 23:16:58.569855 28075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:36437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=32929
--tserver_master_addrs=127.2.76.254:45069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:58.570092 28075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:58.570194 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.570002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:48775)
0809 23:16:58.570138 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:58.570172 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260809 23:16:58.570302 28062 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 23:16:58.570320 28075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:58.574740 28085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:58.574738 28088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:58.574780 28083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:58.575006 28075 server_base.cc:1034] running on GCE node
I20260809 23:16:58.575186 28075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:58.575387 28075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:58.576530 28075 hybrid_clock.cc:648] HybridClock initialized: now 1786317418576511 us; error 28 us; skew 500 ppm
Aug 09 23:16:58 dist-test-slave-7vjv krb5kdc[25913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317418, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:58.578557 28075 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:16:58.578908 28075 webserver.cc:460] Webserver started at http://127.2.76.194:32929/ using document root <none> and password file <none>
I20260809 23:16:58.579151 28075 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:58.579216 28075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:58.580506 28075 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:16:58.581068 28099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:58.581214 28075 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:16:58.581313 28075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "939dac5520824a25bbb09b7375f14c6e"
format_stamp: "Formatted at 2026-08-09 23:16:56 on dist-test-slave-7vjv"
server_key: "1381b3c82653cdf86eab147bd59087f5"
server_key_iv: "0828a7865feb6f6a337551ecb12c289b"
server_key_version: "encryptionkey@0"
I20260809 23:16:58.581606 28075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:58.601523 28075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:58.603072 28075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:58.603219 28075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:58.603410 28075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:58.603688 28075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:58.603734 28075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:58.603761 28075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:58.603785 28075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:58.609045 28075 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:36437
I20260809 23:16:58.609082 28296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:36437 every 8 connection(s)
I20260809 23:16:58.609419 28075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:16:58.610426 28301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.610075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:55719)
0809 23:16:58.610244 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:58.610305 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
W20260809 23:16:58.610797 28298 heartbeater.cc:644] Failed to heartbeat to 127.2.76.254:45069 (0 consecutive failures): Network error: Failed to ping master at 127.2.76.254:45069: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
I20260809 23:16:58.611037 28301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.610952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:34067)
0809 23:16:58.610991 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:58.611021 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 23:16:58.611249 28301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.611203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:41593)
0809 23:16:58.611222 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:58.611239 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260809 23:16:58.611341 28298 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 23:16:58.616151 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 28075
I20260809 23:16:58.616328 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:36841
--local_ip_for_outbound_sockets=127.2.76.195
--tserver_master_addrs=127.2.76.254:45069
--webserver_port=33771
--webserver_interface=127.2.76.195
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:16:58.693763 28302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:16:58.693921 28302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:16:58.693943 28302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:16:58.694852 28302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:16:58.694881 28302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:16:58.694903 28302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:16:58.696044 28302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:36841
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=33771
--tserver_master_addrs=127.2.76.254:45069
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:16:58.696264 28302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:16:58.696466 28302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:16:58.700603 28313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:58.700600 28310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:16:58.700624 28308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:16:58.700785 28302 server_base.cc:1034] running on GCE node
I20260809 23:16:58.701025 28302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:16:58.701246 28302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:16:58.702379 28302 hybrid_clock.cc:648] HybridClock initialized: now 1786317418702368 us; error 26 us; skew 500 ppm
Aug 09 23:16:58 dist-test-slave-7vjv krb5kdc[25913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317418, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:16:58.704517 28302 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:16:58.704934 28302 webserver.cc:460] Webserver started at http://127.2.76.195:33771/ using document root <none> and password file <none>
I20260809 23:16:58.705134 28302 fs_manager.cc:359] Metadata directory not provided
I20260809 23:16:58.705195 28302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:16:58.706333 28302 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:16:58.706811 28325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:58.706944 28302 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:58.707029 28302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "b7f70a187e174dd28553c73cccc1b7d1"
format_stamp: "Formatted at 2026-08-09 23:16:57 on dist-test-slave-7vjv"
server_key: "ec208ad32806b249ad5d693c0a313649"
server_key_iv: "c25f2c01b69e70eefe5785f95fbf345e"
server_key_version: "encryptionkey@0"
I20260809 23:16:58.707295 28302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:16:58.727206 28302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:16:58.728827 28302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:16:58.728960 28302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:16:58.729149 28302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:16:58.729399 28302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:16:58.729446 28302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:58.729480 28302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:16:58.729511 28302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:16:58.735121 28302 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:36841
I20260809 23:16:58.735167 28521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:36841 every 8 connection(s)
I20260809 23:16:58.735435 28302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:16:58.736371 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.735975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:32815)
0809 23:16:58.736160 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:58.736235 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":74,"threads_started":1}
W20260809 23:16:58.736685 28522 heartbeater.cc:644] Failed to heartbeat to 127.2.76.254:45069 (0 consecutive failures): Network error: Failed to ping master at 127.2.76.254:45069: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
I20260809 23:16:58.736928 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.736841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:43025)
0809 23:16:58.736884 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:58.736906 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 23:16:58.737169 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:58.737096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:41929)
0809 23:16:58.737133 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:58.737153 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 23:16:58.737265 28522 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 23:16:58.741350 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 28302
I20260809 23:16:59.570832 28527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:59.570598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:36683)
0809 23:16:59.570737 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:59.570770 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260809 23:16:59.611759 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:59.611588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:60393)
0809 23:16:59.611694 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:59.611722 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":51,"threads_started":1}
I20260809 23:16:59.737776 28530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:16:59.737586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:55139)
0809 23:16:59.737701 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 23:16:59.737733 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260809 23:17:00.215485 28534 raft_consensus.cc:482] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:17:00.215606 28534 raft_consensus.cc:504] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:17:00.215973 28534 leader_election.cc:288] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437), b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841)
Aug 09 23:17:00 dist-test-slave-7vjv krb5kdc[25913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317418, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.195@KRBTEST.COM
Aug 09 23:17:00 dist-test-slave-7vjv krb5kdc[25913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317418, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.194@KRBTEST.COM
I20260809 23:17:00.225615 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:00.216212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:41251 (local address 127.2.76.194:36437)
0809 23:17:00.216396 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:00.216408 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:00.216433 (+ 25us) server_negotiation.cc:407] Connection header received
0809 23:17:00.216884 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:00.216886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:00.216926 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:00.217039 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:17:00.217787 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:00.218525 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:00.219402 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:00.219868 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:00.219994 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:00.222681 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:17:00.222684 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:17:00.222684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:17:00.222692 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:17:00.224018 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:00.224474 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:00.224483 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:00.224484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:00.224567 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:00.224805 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:00.224806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:00.224806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:00.225025 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:17:00.225118 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:00.225419 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 23:17:00.225546 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":55,"threads_started":1}
I20260809 23:17:00.225670 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:00.216090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:36437 (local address 127.2.76.193:41251)
0809 23:17:00.216219 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:00.216229 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:00.216291 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:00.217116 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:00.217119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:00.217499 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:17:00.217670 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:00.217676 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:00.218619 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:00.218619 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:00.219312 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:00.219313 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:00.219970 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:00.219971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:00.220139 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:00.220711 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:17:00.220719 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:17:00.222576 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:17:00.224136 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:00.224137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:00.224137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:00.224384 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:00.224642 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:00.224643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:00.224643 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:00.224734 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:00.225100 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:17:00.225107 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:17:00.225243 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 23:17:00.225344 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 23:17:00.225556 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":24,"thread_start_us":73,"threads_started":1}
I20260809 23:17:00.225948 28542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:00.216156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:36841 (local address 127.2.76.193:50691)
0809 23:17:00.216283 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:00.216292 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:00.216349 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:00.217120 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:00.217121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:00.217496 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:17:00.217670 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:00.217676 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:00.218639 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:00.218640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:00.219313 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:00.219315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:00.219968 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:00.219969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:00.220140 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:00.220759 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:17:00.220772 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:17:00.222202 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:17:00.224036 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:00.224043 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:00.224045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:00.224347 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:00.224616 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:00.224617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:00.224618 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:00.224734 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:00.225157 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:17:00.225158 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:17:00.225243 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 23:17:00.225349 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 23:17:00.225605 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":204,"thread_start_us":63,"threads_started":1}
I20260809 23:17:00.226046 28547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:00.216293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:50691 (local address 127.2.76.195:36841)
0809 23:17:00.216462 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:00.216471 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:00.216499 (+ 28us) server_negotiation.cc:407] Connection header received
0809 23:17:00.216911 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:00.216913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:00.216959 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:00.217114 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:17:00.217784 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:00.218529 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:00.219406 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:00.219864 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:00.220002 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:00.222297 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:17:00.222300 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:17:00.222301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:17:00.222310 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:17:00.223924 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:00.224445 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:00.224448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:00.224448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:00.224543 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:00.224802 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:00.224803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:00.224803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:00.225058 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:17:00.225279 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:00.225789 (+ 510us) server_negotiation.cc:299] Negotiation successful
0809 23:17:00.225921 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
W20260809 23:17:00.226585 27875 leader_election.cc:341] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437): Not found: Tablet not found: 6093ce1bbd29464b8ecaf77ac4323399
W20260809 23:17:00.226678 27868 leader_election.cc:341] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841): Not found: Tablet not found: 6093ce1bbd29464b8ecaf77ac4323399
I20260809 23:17:00.226735 27868 leader_election.cc:302] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2eb4bd3849c484b9b62c1740463113d; no voters: 939dac5520824a25bbb09b7375f14c6e, b7f70a187e174dd28553c73cccc1b7d1
I20260809 23:17:00.226842 28534 raft_consensus.cc:2726] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:17:00.571434 28542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:00.571269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:52851)
0809 23:17:00.571354 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:00.571392 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 23:17:00.612303 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:00.612125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:37989)
0809 23:17:00.612227 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:00.612263 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":49,"threads_started":1}
I20260809 23:17:00.738551 28550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:00.738274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:60995)
0809 23:17:00.738486 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:00.738517 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":62,"threads_started":1}
I20260809 23:17:01.572058 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:01.571853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:56881)
0809 23:17:01.571974 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:01.572011 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":54,"threads_started":1}
I20260809 23:17:01.612785 28558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:01.612612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:49435)
0809 23:17:01.612727 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:01.612754 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":53,"threads_started":1}
I20260809 23:17:01.739329 28560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:01.739138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:37369)
0809 23:17:01.739269 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:01.739297 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260809 23:17:01.740744 28562 raft_consensus.cc:482] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:17:01.740811 28562 raft_consensus.cc:504] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:17:01.740981 28562 leader_election.cc:288] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437), b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841)
W20260809 23:17:01.741387 27868 leader_election.cc:341] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841): Not found: Tablet not found: 6093ce1bbd29464b8ecaf77ac4323399
W20260809 23:17:01.741472 27875 leader_election.cc:341] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437): Not found: Tablet not found: 6093ce1bbd29464b8ecaf77ac4323399
I20260809 23:17:01.741569 27875 leader_election.cc:302] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2eb4bd3849c484b9b62c1740463113d; no voters: 939dac5520824a25bbb09b7375f14c6e, b7f70a187e174dd28553c73cccc1b7d1
I20260809 23:17:01.741660 28562 raft_consensus.cc:2726] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:17:02.572754 28565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:02.572565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:49755)
0809 23:17:02.572680 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:02.572709 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":57,"threads_started":1}
I20260809 23:17:02.613358 28566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:02.613164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:43397)
0809 23:17:02.613282 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:02.613310 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
I20260809 23:17:02.740079 28568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:02.739848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:34717)
0809 23:17:02.740006 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:02.740042 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":72,"threads_started":1}
I20260809 23:17:03.573395 28570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:03.573205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:36183)
0809 23:17:03.573324 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:03.573350 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
I20260809 23:17:03.613878 28571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:03.613720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:59571)
0809 23:17:03.613820 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:03.613845 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":53,"threads_started":1}
I20260809 23:17:03.740670 28572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:03.740465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:46851)
0809 23:17:03.740591 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:03.740635 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260809 23:17:04.370430 28573 raft_consensus.cc:482] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:17:04.370528 28573 raft_consensus.cc:504] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:17:04.370690 28573 leader_election.cc:288] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437), b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841)
W20260809 23:17:04.371073 27875 leader_election.cc:341] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437): Not found: Tablet not found: 6093ce1bbd29464b8ecaf77ac4323399
W20260809 23:17:04.371170 27868 leader_election.cc:341] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841): Not found: Tablet not found: 6093ce1bbd29464b8ecaf77ac4323399
I20260809 23:17:04.371241 27868 leader_election.cc:302] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2eb4bd3849c484b9b62c1740463113d; no voters: 939dac5520824a25bbb09b7375f14c6e, b7f70a187e174dd28553c73cccc1b7d1
I20260809 23:17:04.371308 28573 raft_consensus.cc:2726] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:17:04.573957 28575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:04.573778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:50295)
0809 23:17:04.573885 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:04.573913 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":54,"threads_started":1}
I20260809 23:17:04.614420 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:04.614235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:51347)
0809 23:17:04.614354 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:04.614386 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
I20260809 23:17:04.741271 28577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:04.741090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:60879)
0809 23:17:04.741208 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:04.741237 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260809 23:17:05.574487 28578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:05.574313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:33885)
0809 23:17:05.574417 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:05.574444 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
I20260809 23:17:05.615127 28579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:05.614946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:53905)
0809 23:17:05.615059 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:05.615091 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260809 23:17:05.741811 28580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:05.741640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:43573)
0809 23:17:05.741746 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:05.741777 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":55,"threads_started":1}
I20260809 23:17:06.575194 28581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:06.574985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:36887)
0809 23:17:06.575121 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:06.575148 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260809 23:17:06.615790 28582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:06.615599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:45279)
0809 23:17:06.615725 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:06.615758 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260809 23:17:06.742364 28583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:06.742173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:59419)
0809 23:17:06.742299 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:06.742331 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260809 23:17:07.575820 28584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:07.575634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:42481)
0809 23:17:07.575744 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:07.575775 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260809 23:17:07.616415 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:07.616233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:50807)
0809 23:17:07.616345 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:07.616379 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":54,"threads_started":1}
I20260809 23:17:07.742944 28587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:07.742752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:55817)
0809 23:17:07.742870 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:07.742904 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":59,"threads_started":1}
I20260809 23:17:08.066974 28588 raft_consensus.cc:482] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:17:08.067116 28588 raft_consensus.cc:504] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:17:08.067296 28588 leader_election.cc:288] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437), b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841)
W20260809 23:17:08.067667 27875 leader_election.cc:341] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437): Not found: Tablet not found: 6093ce1bbd29464b8ecaf77ac4323399
W20260809 23:17:08.067734 27868 leader_election.cc:341] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841): Not found: Tablet not found: 6093ce1bbd29464b8ecaf77ac4323399
I20260809 23:17:08.067777 27868 leader_election.cc:302] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2eb4bd3849c484b9b62c1740463113d; no voters: 939dac5520824a25bbb09b7375f14c6e, b7f70a187e174dd28553c73cccc1b7d1
I20260809 23:17:08.067858 28588 raft_consensus.cc:2726] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:17:08.576409 28591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:08.576234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:50185)
0809 23:17:08.576337 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:08.576367 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":54,"threads_started":1}
I20260809 23:17:08.617022 28595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:08.616858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:46455)
0809 23:17:08.616957 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:08.616988 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":51,"threads_started":1}
I20260809 23:17:08.743460 28597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:08.743298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:44727)
0809 23:17:08.743398 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:08.743428 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":53,"threads_started":1}
I20260809 23:17:09.577029 28600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:09.576807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:38735)
0809 23:17:09.576942 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:09.576976 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260809 23:17:09.617712 28601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:09.617550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:55785)
0809 23:17:09.617650 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:09.617680 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":53,"threads_started":1}
I20260809 23:17:09.743957 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:09.743797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:41459)
0809 23:17:09.743896 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:09.743923 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":52,"threads_started":1}
I20260809 23:17:09.865350 28603 raft_consensus.cc:482] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:17:09.865451 28603 raft_consensus.cc:504] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939dac5520824a25bbb09b7375f14c6e" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36437 } } peers { permanent_uuid: "b7f70a187e174dd28553c73cccc1b7d1" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36841 } } peers { permanent_uuid: "a2eb4bd3849c484b9b62c1740463113d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44045 } }
I20260809 23:17:09.865624 28603 leader_election.cc:288] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437), b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841)
W20260809 23:17:09.865983 27875 leader_election.cc:341] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 939dac5520824a25bbb09b7375f14c6e (127.2.76.194:36437): Not found: Tablet not found: 6093ce1bbd29464b8ecaf77ac4323399
W20260809 23:17:09.866070 27868 leader_election.cc:341] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b7f70a187e174dd28553c73cccc1b7d1 (127.2.76.195:36841): Not found: Tablet not found: 6093ce1bbd29464b8ecaf77ac4323399
I20260809 23:17:09.866142 27868 leader_election.cc:302] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2eb4bd3849c484b9b62c1740463113d; no voters: 939dac5520824a25bbb09b7375f14c6e, b7f70a187e174dd28553c73cccc1b7d1
I20260809 23:17:09.866205 28603 raft_consensus.cc:2726] T 6093ce1bbd29464b8ecaf77ac4323399 P a2eb4bd3849c484b9b62c1740463113d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:17:10.577605 28604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:10.577401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:37501)
0809 23:17:10.577528 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:10.577559 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260809 23:17:10.618335 28608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:10.618162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:53347)
0809 23:17:10.618269 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:10.618301 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":55,"threads_started":1}
I20260809 23:17:10.744489 28609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:10.744325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:40309)
0809 23:17:10.744427 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:10.744457 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":50,"threads_started":1}
I20260809 23:17:11.578270 28610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:11.578083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:51999)
0809 23:17:11.578196 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:11.578225 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260809 23:17:11.618857 28612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:11.618671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:32979)
0809 23:17:11.618793 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:11.618824 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
I20260809 23:17:11.745060 28614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:11.744863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:55973)
0809 23:17:11.744988 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:11.745027 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260809 23:17:12.578876 28616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:12.578654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:49865)
0809 23:17:12.578782 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:12.578811 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260809 23:17:12.619392 28618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:12.619225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:58249)
0809 23:17:12.619330 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:12.619357 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":56,"threads_started":1}
I20260809 23:17:12.745668 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:12.745492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:57457)
0809 23:17:12.745602 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:12.745632 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260809 23:17:13.579742 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:13.579444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.193:32837)
0809 23:17:13.579662 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:13.579696 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":51,"threads_started":1}
I20260809 23:17:13.620128 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:13.619922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.194:57953)
0809 23:17:13.620030 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:13.620059 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260809 23:17:13.746431 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:13.746213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45069 (local address 127.2.76.195:55447)
0809 23:17:13.746365 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:13.746396 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260809 23:17:13.746791 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 27795
I20260809 23:17:13.759876 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 28075
I20260809 23:17:13.766317 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 28302
2026-08-09T23:17:13Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18494 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:17:13 dist-test-slave-7vjv krb5kdc[28627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:17:13 dist-test-slave-7vjv krb5kdc[28627](info): set up 2 sockets
Aug 09 23:17:13 dist-test-slave-7vjv krb5kdc[28627](info): commencing operation
krb5kdc: starting...
W20260809 23:17:14.916455 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:17:14 dist-test-slave-7vjv krb5kdc[28627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:17:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:17:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:17:15.022419 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45141
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:45141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:17:15.094151 28658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:17:15.094355 28658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:17:15.094390 28658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:17:15.095420 28658 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 23:17:15.095477 28658 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 23:17:15.095502 28658 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 23:17:15.095535 28658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:17:15.095557 28658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:17:15.095584 28658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:17:15.095606 28658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:17:15.095630 28658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:17:15.097004 28658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:45141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:17:15.097249 28658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:17:15.097442 28658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:17:15.102010 28668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:15.102034 28667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:15.102015 28673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:15.102236 28658 server_base.cc:1034] running on GCE node
I20260809 23:17:15.102419 28658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:17:15.102662 28658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:17:15.103783 28658 hybrid_clock.cc:648] HybridClock initialized: now 1786317435103768 us; error 31 us; skew 500 ppm
Aug 09 23:17:15 dist-test-slave-7vjv krb5kdc[28627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317435, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:17:15.105955 28658 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:17:15.106344 28658 webserver.cc:460] Webserver started at http://127.2.76.254:37975/ using document root <none> and password file <none>
I20260809 23:17:15.106544 28658 fs_manager.cc:359] Metadata directory not provided
I20260809 23:17:15.106602 28658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:17:15.106724 28658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:17:15.107558 28658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "053303cba11b488fb38a9823bb381dc5"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "8aa6cab073b70ba113f80d6e9e58d5c5"
server_key_iv: "c98198bc0cdb10ecaaab284fe4ef5dee"
server_key_version: "encryptionkey@0"
I20260809 23:17:15.107883 28658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "053303cba11b488fb38a9823bb381dc5"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "8aa6cab073b70ba113f80d6e9e58d5c5"
server_key_iv: "c98198bc0cdb10ecaaab284fe4ef5dee"
server_key_version: "encryptionkey@0"
I20260809 23:17:15.109081 28658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 23:17:15.109661 28682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:15.109829 28658 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:15.109920 28658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "053303cba11b488fb38a9823bb381dc5"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "8aa6cab073b70ba113f80d6e9e58d5c5"
server_key_iv: "c98198bc0cdb10ecaaab284fe4ef5dee"
server_key_version: "encryptionkey@0"
I20260809 23:17:15.110011 28658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:17:15.116320 28658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:17:15.117735 28658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:17:15.117866 28658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:17:15.121534 28658 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:45141
I20260809 23:17:15.121562 28767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:45141 every 8 connection(s)
I20260809 23:17:15.121840 28658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:17:15.122505 28769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:17:15.124313 28769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Bootstrap starting.
I20260809 23:17:15.124771 28769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Neither blocks nor log segments found. Creating new log.
I20260809 23:17:15.124977 28769 log.cc:824] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Log is configured to *not* fsync() on all Append() calls
I20260809 23:17:15.125512 28769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: No bootstrap required, opened a new log
I20260809 23:17:15.126418 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 28658
I20260809 23:17:15.126514 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:17:15.126689 2355 external_mini_cluster.cc:1428] Setting key a08ce09a599d218b39d22744b472ffef
I20260809 23:17:15.127252 28769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } }
I20260809 23:17:15.127384 28769 raft_consensus.cc:374] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:17:15.127405 28769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053303cba11b488fb38a9823bb381dc5, State: Initialized, Role: FOLLOWER
I20260809 23:17:15.127525 28769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } }
I20260809 23:17:15.127609 28769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:17:15.127637 28769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:17:15.127717 28769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:17:15.128639 28769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } }
I20260809 23:17:15.128715 28769 leader_election.cc:302] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 053303cba11b488fb38a9823bb381dc5; no voters:
I20260809 23:17:15.128866 28769 leader_election.cc:288] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:17:15.128893 28781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:17:15.128980 28781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 1 LEADER]: Becoming Leader. State: Replica: 053303cba11b488fb38a9823bb381dc5, State: Running, Role: LEADER
I20260809 23:17:15.129096 28769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:17:15.129102 28781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } }
I20260809 23:17:15.129485 28782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "053303cba11b488fb38a9823bb381dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } } }
I20260809 23:17:15.129606 28782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [sys.catalog]: This master's current role is: LEADER
I20260809 23:17:15.129523 28783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 053303cba11b488fb38a9823bb381dc5. Latest consensus state: current_term: 1 leader_uuid: "053303cba11b488fb38a9823bb381dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } } }
I20260809 23:17:15.129761 28783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [sys.catalog]: This master's current role is: LEADER
W20260809 23:17:15.130978 28802 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:17:15.131047 28802 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:17:15.131098 28788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:17:15.131244 28788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:17:15.132624 28788 catalog_manager.cc:1347] Generated new cluster ID: dbd9bbb4f4644c60b3919609d40756d2
I20260809 23:17:15.132673 28788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:17:15 dist-test-slave-7vjv krb5kdc[28627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:17:15.135916 28780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.127139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47522 (local address 127.2.76.254:45141)
0809 23:17:15.127304 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:15.127317 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:15.127338 (+ 21us) server_negotiation.cc:407] Connection header received
0809 23:17:15.127439 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:15.127445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:15.127490 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:15.127630 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:17:15.128335 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.129386 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.130442 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.131012 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.131402 (+ 390us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.133406 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:17:15.133409 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:17:15.133409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:17:15.133417 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:17:15.134686 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:15.135013 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:15.135015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:15.135015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:15.135101 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:15.135264 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:15.135264 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:15.135265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:15.135441 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:17:15.135541 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:15.135631 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 23:17:15.135715 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":99,"threads_started":1}
I20260809 23:17:15.141172 28788 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:17:15.141606 28788 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:17:15.147707 28788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Generated new TSK 0
I20260809 23:17:15.147850 28788 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:17:15.178434 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45141
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:17:15.249533 28811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:17:15.249724 28811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:17:15.249759 28811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:17:15.250730 28811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:17:15.250787 28811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:17:15.250825 28811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:17:15.252537 28811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:17:15.252779 28811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:17:15.253042 28811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:17:15.257807 28811 server_base.cc:1034] running on GCE node
W20260809 23:17:15.257777 28822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:15.257777 28819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:15.257921 28826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:15.258165 28811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:17:15.258391 28811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:17:15.259547 28811 hybrid_clock.cc:648] HybridClock initialized: now 1786317435259531 us; error 26 us; skew 500 ppm
Aug 09 23:17:15 dist-test-slave-7vjv krb5kdc[28627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317435, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:17:15.261742 28811 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:17:15.262214 28811 webserver.cc:460] Webserver started at http://127.2.76.193:37363/ using document root <none> and password file <none>
I20260809 23:17:15.262454 28811 fs_manager.cc:359] Metadata directory not provided
I20260809 23:17:15.262517 28811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:17:15.262658 28811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:17:15.263651 28811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "f1508c984feb44458692d0d59c45257d"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "38a5bd66fafb0b68c4705e020b5399e8"
server_key_iv: "f5e256cf8b56d8ef11e98d5eea0e8c0d"
server_key_version: "encryptionkey@0"
I20260809 23:17:15.264071 28811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "f1508c984feb44458692d0d59c45257d"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "38a5bd66fafb0b68c4705e020b5399e8"
server_key_iv: "f5e256cf8b56d8ef11e98d5eea0e8c0d"
server_key_version: "encryptionkey@0"
I20260809 23:17:15.265494 28811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:17:15.266209 28837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:15.266453 28811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:17:15.266598 28811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "f1508c984feb44458692d0d59c45257d"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "38a5bd66fafb0b68c4705e020b5399e8"
server_key_iv: "f5e256cf8b56d8ef11e98d5eea0e8c0d"
server_key_version: "encryptionkey@0"
I20260809 23:17:15.266703 28811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:17:15.281692 28811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:17:15.283245 28811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:17:15.283386 28811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:17:15.283593 28811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:17:15.283855 28811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:17:15.283905 28811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:15.283933 28811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:17:15.283957 28811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:15.289304 28811 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:40647
I20260809 23:17:15.289347 29035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:40647 every 8 connection(s)
I20260809 23:17:15.289613 28811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:17:15.292742 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 28811
I20260809 23:17:15.292821 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:17:15.292981 2355 external_mini_cluster.cc:1428] Setting key 128f974cd0d12142ee5a74282179b3c2
Aug 09 23:17:15 dist-test-slave-7vjv krb5kdc[28627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317435, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:17:15.298090 28780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.290286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:46843 (local address 127.2.76.254:45141)
0809 23:17:15.290345 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:15.290347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:15.290554 (+ 207us) server_negotiation.cc:407] Connection header received
0809 23:17:15.290932 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:15.290933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:15.290948 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:15.291012 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:17:15.291755 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.292267 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.293191 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.293404 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.293501 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.295538 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:17:15.295539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:17:15.295540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:17:15.295547 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:17:15.296867 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:15.297237 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:15.297238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:15.297238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:15.297290 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:15.297533 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:15.297534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:15.297534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:15.297635 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:17:15.297707 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:15.297936 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 23:17:15.298009 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 23:17:15.298247 29042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.290207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:46843)
0809 23:17:15.290428 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:15.290464 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:15.290843 (+ 379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:15.291057 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:15.291059 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:15.291537 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:17:15.291690 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.291693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.292337 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.292337 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.293104 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.293107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.293543 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.293545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.293684 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.294177 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:17:15.294189 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:17:15.295444 (+ 1255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:17:15.296959 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:15.296966 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:15.296967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:15.297166 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:15.297362 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:15.297363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:15.297363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:15.297461 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:15.297699 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:17:15.297706 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:17:15.297823 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 23:17:15.297915 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 23:17:15.298057 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":92,"threads_started":1}
I20260809 23:17:15.298765 29039 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45141
I20260809 23:17:15.298841 29039 heartbeater.cc:464] Registering TS with master...
I20260809 23:17:15.299013 29039 heartbeater.cc:511] Master 127.2.76.254:45141 requested a full tablet report, sending...
I20260809 23:17:15.299427 28707 ts_manager.cc:194] Registered new tserver with Master: f1508c984feb44458692d0d59c45257d (127.2.76.193:40647)
I20260809 23:17:15.299937 28707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:46843
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:17:15.316854 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45141
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:17:15.389691 29045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:17:15.389848 29045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:17:15.389868 29045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:17:15.390911 29045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:17:15.390954 29045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:17:15.390977 29045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:17:15.392297 29045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:17:15.392513 29045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:17:15.392720 29045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:17:15.397244 29067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:15.397281 29045 server_base.cc:1034] running on GCE node
W20260809 23:17:15.397261 29060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:15.397253 29058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:15.397673 29045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:17:15.397897 29045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:17:15.399051 29045 hybrid_clock.cc:648] HybridClock initialized: now 1786317435399042 us; error 36 us; skew 500 ppm
Aug 09 23:17:15 dist-test-slave-7vjv krb5kdc[28627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317435, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:17:15.401451 29045 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:17:15.401827 29045 webserver.cc:460] Webserver started at http://127.2.76.194:41207/ using document root <none> and password file <none>
I20260809 23:17:15.402030 29045 fs_manager.cc:359] Metadata directory not provided
I20260809 23:17:15.402093 29045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:17:15.402236 29045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:17:15.403201 29045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "968a3fdff4a94c588859f9e93bd115b7"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "ac4dafe8a1d877298a20bb3399f674ca"
server_key_iv: "0f3388828e110b2c28f773b99bb2df9c"
server_key_version: "encryptionkey@0"
I20260809 23:17:15.403566 29045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "968a3fdff4a94c588859f9e93bd115b7"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "ac4dafe8a1d877298a20bb3399f674ca"
server_key_iv: "0f3388828e110b2c28f773b99bb2df9c"
server_key_version: "encryptionkey@0"
I20260809 23:17:15.404785 29045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 23:17:15.405427 29081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:15.405583 29045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:15.405675 29045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "968a3fdff4a94c588859f9e93bd115b7"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "ac4dafe8a1d877298a20bb3399f674ca"
server_key_iv: "0f3388828e110b2c28f773b99bb2df9c"
server_key_version: "encryptionkey@0"
I20260809 23:17:15.405746 29045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:17:15.417569 29045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:17:15.419816 29045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:17:15.419956 29045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:17:15.420233 29045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:17:15.420495 29045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:17:15.420545 29045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:15.420575 29045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:17:15.420598 29045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:15.425755 29045 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:43841
I20260809 23:17:15.425802 29281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:43841 every 8 connection(s)
I20260809 23:17:15.426060 29045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:17:15.431097 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 29045
I20260809 23:17:15.431239 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:17:15.431427 2355 external_mini_cluster.cc:1428] Setting key 866785c28bf25d03a00a9119b3dc5ee0
Aug 09 23:17:15 dist-test-slave-7vjv krb5kdc[28627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317435, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:17:15.434523 28780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.426649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:33475 (local address 127.2.76.254:45141)
0809 23:17:15.426716 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:15.426718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:15.426942 (+ 224us) server_negotiation.cc:407] Connection header received
0809 23:17:15.427337 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:15.427338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:15.427347 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:15.427391 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:17:15.428223 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.428666 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.429560 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.429766 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.429888 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.432021 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:17:15.432023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:17:15.432023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:17:15.432030 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:17:15.433317 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:15.433678 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:15.433679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:15.433680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:15.433732 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:15.433960 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:15.433961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:15.433961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:15.434072 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:17:15.434135 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:15.434374 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 23:17:15.434444 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:17:15.434682 29288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.426626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:33475)
0809 23:17:15.426800 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:15.426848 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:15.427258 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:15.427442 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:15.427445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:15.427931 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:17:15.428136 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.428142 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.428749 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.428749 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.429487 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.429489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.429934 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.429935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.430035 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.430458 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:17:15.430466 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:17:15.431924 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:17:15.433395 (+ 1471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:15.433401 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:15.433403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:15.433606 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:15.433805 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:15.433806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:15.433806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:15.433901 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:15.434140 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:17:15.434146 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:17:15.434261 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 23:17:15.434353 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 23:17:15.434484 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260809 23:17:15.435114 29283 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45141
I20260809 23:17:15.435222 29283 heartbeater.cc:464] Registering TS with master...
I20260809 23:17:15.435380 29283 heartbeater.cc:511] Master 127.2.76.254:45141 requested a full tablet report, sending...
I20260809 23:17:15.435753 28707 ts_manager.cc:194] Registered new tserver with Master: 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841)
I20260809 23:17:15.436280 28707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:33475
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:17:15.454604 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45141
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:17:15.527108 29294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:17:15.527310 29294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:17:15.527333 29294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:17:15.528393 29294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:17:15.528438 29294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:17:15.528462 29294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:17:15.529831 29294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:45141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:17:15.529989 29294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:17:15.530177 29294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:17:15.534480 29308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:15.534487 29302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:15.534487 29304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:15.534607 29294 server_base.cc:1034] running on GCE node
I20260809 23:17:15.534837 29294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:17:15.535055 29294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:17:15.536151 29294 hybrid_clock.cc:648] HybridClock initialized: now 1786317435536133 us; error 33 us; skew 500 ppm
Aug 09 23:17:15 dist-test-slave-7vjv krb5kdc[28627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317435, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:17:15.538362 29294 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:17:15.538758 29294 webserver.cc:460] Webserver started at http://127.2.76.195:44641/ using document root <none> and password file <none>
I20260809 23:17:15.538954 29294 fs_manager.cc:359] Metadata directory not provided
I20260809 23:17:15.539019 29294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:17:15.539168 29294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:17:15.540059 29294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "108c8e78f0c64c3b9295f7f9991232ba"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "013e0fe73180527fe6504179c3ebe2af"
server_key_iv: "4e514ac7c3b6d484a680918153a0a5a2"
server_key_version: "encryptionkey@0"
I20260809 23:17:15.540444 29294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "108c8e78f0c64c3b9295f7f9991232ba"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "013e0fe73180527fe6504179c3ebe2af"
server_key_iv: "4e514ac7c3b6d484a680918153a0a5a2"
server_key_version: "encryptionkey@0"
I20260809 23:17:15.541613 29294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:17:15.542305 29322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:15.542461 29294 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:17:15.542557 29294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "108c8e78f0c64c3b9295f7f9991232ba"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "013e0fe73180527fe6504179c3ebe2af"
server_key_iv: "4e514ac7c3b6d484a680918153a0a5a2"
server_key_version: "encryptionkey@0"
I20260809 23:17:15.542631 29294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:17:15.551736 29294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:17:15.553958 29294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:17:15.554100 29294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:17:15.554286 29294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:17:15.554543 29294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:17:15.554597 29294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:15.554631 29294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:17:15.554654 29294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:15.560025 29294 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:39451
I20260809 23:17:15.560067 29510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:39451 every 8 connection(s)
I20260809 23:17:15.560335 29294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:17:15 dist-test-slave-7vjv krb5kdc[28627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317435, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:17:15.568135 28780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.560933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:45381 (local address 127.2.76.254:45141)
0809 23:17:15.560974 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:15.560976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:15.561155 (+ 179us) server_negotiation.cc:407] Connection header received
0809 23:17:15.561574 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:15.561575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:15.561587 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:15.561632 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:17:15.562447 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.562918 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.563886 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.564072 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.564171 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.565904 (+ 1733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:17:15.565906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:17:15.565906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:17:15.565913 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:17:15.566927 (+ 1014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:15.567270 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:15.567271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:15.567271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:15.567334 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:15.567552 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:15.567553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:15.567553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:15.567654 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:17:15.567715 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:15.567959 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 23:17:15.568034 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:17:15.568264 29516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.560894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:45381)
0809 23:17:15.561039 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:15.561080 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:15.561473 (+ 393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:15.561686 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:15.561689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:15.562191 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:17:15.562349 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.562353 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.563012 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.563013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.563793 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.563795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.564183 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.564184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.564295 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.564641 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:17:15.564648 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:17:15.565801 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:17:15.567000 (+ 1199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:15.567008 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:15.567010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:15.567199 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:15.567395 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:15.567396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:15.567396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:15.567489 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:15.567718 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:17:15.567725 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:17:15.567842 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 23:17:15.567929 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 23:17:15.568059 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":56,"threads_started":1}
I20260809 23:17:15.568600 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 29294
I20260809 23:17:15.568707 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:17:15.568709 29513 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45141
I20260809 23:17:15.568782 29513 heartbeater.cc:464] Registering TS with master...
I20260809 23:17:15.568940 2355 external_mini_cluster.cc:1428] Setting key 2b1425cd1baa7855cc7a6b53e9c1c885
I20260809 23:17:15.568946 29513 heartbeater.cc:511] Master 127.2.76.254:45141 requested a full tablet report, sending...
I20260809 23:17:15.569304 28706 ts_manager.cc:194] Registered new tserver with Master: 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451)
I20260809 23:17:15.569708 28706 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:45381
I20260809 23:17:15.569715 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:17:15.575835 28780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.571042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47536 (local address 127.2.76.254:45141)
0809 23:17:15.571109 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:15.571111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:15.571215 (+ 104us) server_negotiation.cc:407] Connection header received
0809 23:17:15.571247 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:15.571248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:15.571255 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:15.571294 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:17:15.571782 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.572247 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.572861 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.573004 (+ 143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.573049 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.573928 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:17:15.573929 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:17:15.573929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:17:15.573934 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:17:15.575084 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:15.575347 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:15.575348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:15.575348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:15.575392 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:15.575493 (+ 101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:15.575493 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:15.575493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:15.575599 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:17:15.575660 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:15.575743 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 23:17:15.575791 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 23:17:15.577342 28706 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:17:15.578685 28706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:17:15.584160 29526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.580550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:39451 (local address 127.0.0.1:34454)
0809 23:17:15.580799 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:15.580808 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:15.580862 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:15.581069 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:15.581071 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:15.581073 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:17:15.581133 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.581135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.582079 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.582081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.583113 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.583115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.583864 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.583865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.583936 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.583948 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 23:17:15.584025 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 23:17:15.584076 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":110,"threads_started":1}
I20260809 23:17:15.584169 29533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.580741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34454 (local address 127.2.76.195:39451)
0809 23:17:15.580937 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:15.580941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:15.580950 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:17:15.580979 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:15.580981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:15.580993 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:15.581065 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:17:15.581204 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.581977 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.583203 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.583773 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.583835 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.583879 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:15.584048 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 23:17:15.584081 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":105,"threads_started":1}
I20260809 23:17:15.584529 29534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.580804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44706 (local address 127.2.76.194:43841)
0809 23:17:15.581002 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:15.581006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:15.581016 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:17:15.581055 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:15.581057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:15.581070 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:15.581145 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:17:15.581314 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.582325 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.583460 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.583964 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.584019 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.584065 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:15.584377 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 23:17:15.584436 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":81,"threads_started":1}
I20260809 23:17:15.584545 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.580521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:40647 (local address 127.0.0.1:48220)
0809 23:17:15.580706 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:15.580742 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:15.580825 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:15.581541 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:15.581542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:15.581544 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:17:15.581579 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.581580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.582653 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.582654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.583397 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.583399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.584297 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.584298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.584360 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.584363 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:17:15.584445 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 23:17:15.584485 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":53,"threads_started":1}
I20260809 23:17:15.584322 29530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.580550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:43841 (local address 127.0.0.1:44706)
0809 23:17:15.580939 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:15.580948 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:15.580987 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:15.581189 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:15.581191 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:15.581193 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:17:15.581237 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.581239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.582426 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.582427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.583380 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.583382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.584096 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.584097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.584163 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.584167 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:17:15.584212 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 23:17:15.584251 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":12,"thread_start_us":101,"threads_started":1}
I20260809 23:17:15.584743 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.580615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48220 (local address 127.2.76.193:40647)
0809 23:17:15.581323 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:15.581327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:15.581335 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:17:15.581372 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:15.581374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:15.581388 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:15.581458 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:17:15.581645 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.582560 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.583483 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.584150 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.584211 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.584264 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:15.584464 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 23:17:15.584504 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":94,"threads_started":1}
I20260809 23:17:15.585093 29401 tablet_service.cc:1467] Processing CreateTablet for tablet 9408e3ffed654912bfd977a949561582 (DEFAULT_TABLE table=test-table [id=d19726722ba84bd194d1ba77be20e06a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:17:15.585327 29163 tablet_service.cc:1467] Processing CreateTablet for tablet 9408e3ffed654912bfd977a949561582 (DEFAULT_TABLE table=test-table [id=d19726722ba84bd194d1ba77be20e06a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:17:15.585434 29401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9408e3ffed654912bfd977a949561582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:17:15.585453 28918 tablet_service.cc:1467] Processing CreateTablet for tablet 9408e3ffed654912bfd977a949561582 (DEFAULT_TABLE table=test-table [id=d19726722ba84bd194d1ba77be20e06a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:17:15.585657 29163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9408e3ffed654912bfd977a949561582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:17:15.585726 28918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9408e3ffed654912bfd977a949561582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:17:15.587586 29535 tablet_bootstrap.cc:492] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba: Bootstrap starting.
I20260809 23:17:15.588184 29535 tablet_bootstrap.cc:654] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba: Neither blocks nor log segments found. Creating new log.
I20260809 23:17:15.588568 29539 tablet_bootstrap.cc:492] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7: Bootstrap starting.
I20260809 23:17:15.588752 29535 log.cc:824] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba: Log is configured to *not* fsync() on all Append() calls
I20260809 23:17:15.589221 29539 tablet_bootstrap.cc:654] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7: Neither blocks nor log segments found. Creating new log.
I20260809 23:17:15.589450 29535 tablet_bootstrap.cc:492] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba: No bootstrap required, opened a new log
I20260809 23:17:15.589529 29535 ts_tablet_manager.cc:1397] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:17:15.589798 29539 log.cc:824] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7: Log is configured to *not* fsync() on all Append() calls
I20260809 23:17:15.590137 29537 tablet_bootstrap.cc:492] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: Bootstrap starting.
I20260809 23:17:15.590476 29539 tablet_bootstrap.cc:492] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7: No bootstrap required, opened a new log
I20260809 23:17:15.590569 29539 ts_tablet_manager.cc:1397] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:17:15.590864 29537 tablet_bootstrap.cc:654] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: Neither blocks nor log segments found. Creating new log.
I20260809 23:17:15.591425 29537 log.cc:824] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:17:15.591657 29535 raft_consensus.cc:348] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } } peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } }
I20260809 23:17:15.591840 29535 raft_consensus.cc:374] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:17:15.591897 29535 raft_consensus.cc:729] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 108c8e78f0c64c3b9295f7f9991232ba, State: Initialized, Role: FOLLOWER
I20260809 23:17:15.592034 29535 consensus_queue.cc:260] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } } peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } }
I20260809 23:17:15.592211 29537 tablet_bootstrap.cc:492] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: No bootstrap required, opened a new log
I20260809 23:17:15.592275 29537 ts_tablet_manager.cc:1397] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:17:15.592312 29535 ts_tablet_manager.cc:1428] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 23:17:15.592422 29513 heartbeater.cc:503] Master 127.2.76.254:45141 was elected leader, sending a full tablet report...
I20260809 23:17:15.592327 29539 raft_consensus.cc:348] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } } peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } }
I20260809 23:17:15.592461 29539 raft_consensus.cc:374] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:17:15.592489 29539 raft_consensus.cc:729] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 968a3fdff4a94c588859f9e93bd115b7, State: Initialized, Role: FOLLOWER
I20260809 23:17:15.592597 29539 consensus_queue.cc:260] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } } peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } }
I20260809 23:17:15.592825 29539 ts_tablet_manager.cc:1428] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:17:15.592859 29283 heartbeater.cc:503] Master 127.2.76.254:45141 was elected leader, sending a full tablet report...
I20260809 23:17:15.594072 29537 raft_consensus.cc:348] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } } peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } }
I20260809 23:17:15.594194 29537 raft_consensus.cc:374] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:17:15.594211 29537 raft_consensus.cc:729] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1508c984feb44458692d0d59c45257d, State: Initialized, Role: FOLLOWER
I20260809 23:17:15.594300 29537 consensus_queue.cc:260] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } } peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } }
I20260809 23:17:15.594549 29537 ts_tablet_manager.cc:1428] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:17:15.594585 29039 heartbeater.cc:503] Master 127.2.76.254:45141 was elected leader, sending a full tablet report...
I20260809 23:17:15.644203 29554 raft_consensus.cc:482] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:17:15.644272 29554 raft_consensus.cc:504] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } } peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } }
I20260809 23:17:15.644599 29554 leader_election.cc:288] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1508c984feb44458692d0d59c45257d (127.2.76.193:40647), 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451)
I20260809 23:17:15.648002 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.644756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:50215 (local address 127.2.76.193:40647)
0809 23:17:15.644788 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:15.644790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:15.644824 (+ 34us) server_negotiation.cc:407] Connection header received
0809 23:17:15.644889 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:15.644892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:15.644905 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:15.644975 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:17:15.645129 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.645754 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.647094 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.647657 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.647727 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.647733 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:15.647910 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 23:17:15.647943 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:17:15.648029 29533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.644801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:36225 (local address 127.2.76.195:39451)
0809 23:17:15.644851 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:15.644853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:15.644942 (+ 89us) server_negotiation.cc:407] Connection header received
0809 23:17:15.644999 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:15.645000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:15.645007 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:15.645050 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:17:15.645176 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.645765 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.647094 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.647650 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.647726 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.647733 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:15.647910 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 23:17:15.647943 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 23:17:15.648030 29288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.644721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:40647 (local address 127.2.76.194:50215)
0809 23:17:15.644776 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:15.644786 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:15.644838 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:15.644986 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:15.644987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:15.644988 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:17:15.645072 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.645073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.645828 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.645829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.647003 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.647006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.647740 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.647741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.647825 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.647833 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 23:17:15.647891 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 23:17:15.647938 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":88}
I20260809 23:17:15.648038 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.644743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:39451 (local address 127.2.76.194:36225)
0809 23:17:15.644882 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:15.644891 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:15.644933 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:15.645065 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:15.645066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:15.645066 (+ 0us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:17:15.645130 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.645132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.645828 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.645829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.647002 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:15.647006 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.647740 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.647741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:15.647825 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.647834 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 23:17:15.647891 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 23:17:15.647937 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":39,"thread_start_us":72,"threads_started":1}
I20260809 23:17:15.648411 28948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9408e3ffed654912bfd977a949561582" candidate_uuid: "968a3fdff4a94c588859f9e93bd115b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1508c984feb44458692d0d59c45257d" is_pre_election: true
I20260809 23:17:15.648510 29434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9408e3ffed654912bfd977a949561582" candidate_uuid: "968a3fdff4a94c588859f9e93bd115b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "108c8e78f0c64c3b9295f7f9991232ba" is_pre_election: true
I20260809 23:17:15.648550 28948 raft_consensus.cc:2454] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 968a3fdff4a94c588859f9e93bd115b7 in term 0.
I20260809 23:17:15.648623 29434 raft_consensus.cc:2454] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 968a3fdff4a94c588859f9e93bd115b7 in term 0.
I20260809 23:17:15.648731 29087 leader_election.cc:302] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 968a3fdff4a94c588859f9e93bd115b7, f1508c984feb44458692d0d59c45257d; no voters:
I20260809 23:17:15.648846 29554 raft_consensus.cc:2781] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:17:15.648896 29554 raft_consensus.cc:482] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:17:15.648916 29554 raft_consensus.cc:3037] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:17:15.649426 29554 raft_consensus.cc:504] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } } peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } }
I20260809 23:17:15.649554 29554 leader_election.cc:288] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [CANDIDATE]: Term 1 election: Requested vote from peers f1508c984feb44458692d0d59c45257d (127.2.76.193:40647), 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451)
I20260809 23:17:15.649695 28948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9408e3ffed654912bfd977a949561582" candidate_uuid: "968a3fdff4a94c588859f9e93bd115b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1508c984feb44458692d0d59c45257d"
I20260809 23:17:15.649691 29434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9408e3ffed654912bfd977a949561582" candidate_uuid: "968a3fdff4a94c588859f9e93bd115b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "108c8e78f0c64c3b9295f7f9991232ba"
I20260809 23:17:15.649803 28948 raft_consensus.cc:3037] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:17:15.649829 29434 raft_consensus.cc:3037] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:17:15.650417 28948 raft_consensus.cc:2454] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 968a3fdff4a94c588859f9e93bd115b7 in term 1.
I20260809 23:17:15.650518 29434 raft_consensus.cc:2454] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 968a3fdff4a94c588859f9e93bd115b7 in term 1.
I20260809 23:17:15.650583 29087 leader_election.cc:302] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 968a3fdff4a94c588859f9e93bd115b7, f1508c984feb44458692d0d59c45257d; no voters:
I20260809 23:17:15.650672 29554 raft_consensus.cc:2781] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:17:15.650794 29554 raft_consensus.cc:686] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [term 1 LEADER]: Becoming Leader. State: Replica: 968a3fdff4a94c588859f9e93bd115b7, State: Running, Role: LEADER
I20260809 23:17:15.650884 29554 consensus_queue.cc:237] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } } peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } }
I20260809 23:17:15.651598 28707 catalog_manager.cc:5495] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194). New cstate: current_term: 1 leader_uuid: "968a3fdff4a94c588859f9e93bd115b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } health_report { overall_health: HEALTHY } } }
I20260809 23:17:15.658811 29534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.657105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44716 (local address 127.2.76.194:43841)
0809 23:17:15.657134 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:15.657136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:15.657143 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:17:15.657218 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:15.657218 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:15.657230 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:15.657288 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:17:15.657372 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.657736 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.658310 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:15.658454 (+ 144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:15.658495 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:15.658620 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:17:15.658688 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:17:15.658735 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:15.658749 (+ 14us) server_negotiation.cc:299] Negotiation successful
0809 23:17:15.658761 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 23:17:15.660135 29434 raft_consensus.cc:1261] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba [term 1 FOLLOWER]: Refusing update from remote peer 968a3fdff4a94c588859f9e93bd115b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:17:15.660158 28948 raft_consensus.cc:1261] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Refusing update from remote peer 968a3fdff4a94c588859f9e93bd115b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:17:15.660444 29557 consensus_queue.cc:1035] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:17:15.660542 29554 consensus_queue.cc:1035] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:17:15.662945 29562 mvcc.cc:205] Tried to move back new op lower bound from 7316756216461746176 to 7316756216426770432. Current Snapshot: MvccSnapshot[applied={T|T < 7316756216461746176}]
I20260809 23:17:15.663350 29561 mvcc.cc:205] Tried to move back new op lower bound from 7316756216461746176 to 7316756216426770432. Current Snapshot: MvccSnapshot[applied={T|T < 7316756216461746176 or (T in {7316756216461746176})}]
I20260809 23:17:15.663403 29563 mvcc.cc:205] Tried to move back new op lower bound from 7316756216461746176 to 7316756216426770432. Current Snapshot: MvccSnapshot[applied={T|T < 7316756216461746176}]
I20260809 23:17:15.664676 28706 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:17:15.664754 28706 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:17:15.665323 28706 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Sending DeleteTablet for 3 replicas of tablet 9408e3ffed654912bfd977a949561582
I20260809 23:17:15.665609 29163 tablet_service.cc:1514] Processing DeleteTablet for tablet 9408e3ffed654912bfd977a949561582 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:17:15 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:44706
I20260809 23:17:15.665597 28918 tablet_service.cc:1514] Processing DeleteTablet for tablet 9408e3ffed654912bfd977a949561582 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:17:15 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:48220
I20260809 23:17:15.665635 29401 tablet_service.cc:1514] Processing DeleteTablet for tablet 9408e3ffed654912bfd977a949561582 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:17:15 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:34454
I20260809 23:17:15.665865 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 28811
I20260809 23:17:15.676177 29577 tablet_replica.cc:330] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7: stopping tablet replica
I20260809 23:17:15.676316 29578 tablet_replica.cc:330] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba: stopping tablet replica
I20260809 23:17:15.676498 29578 raft_consensus.cc:2229] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:17:15.676544 29577 raft_consensus.cc:2229] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:17:15.676677 29577 raft_consensus.cc:2258] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:17:15.676749 29578 raft_consensus.cc:2258] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:17:15.677686 29578 ts_tablet_manager.cc:1905] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:17:15.677788 29577 ts_tablet_manager.cc:1905] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:17:15.679266 29578 ts_tablet_manager.cc:1918] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:17:15.679417 29578 log.cc:1196] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/9408e3ffed654912bfd977a949561582
I20260809 23:17:15.679782 29578 ts_tablet_manager.cc:1939] T 9408e3ffed654912bfd977a949561582 P 108c8e78f0c64c3b9295f7f9991232ba: Deleting consensus metadata
I20260809 23:17:15.679898 29577 ts_tablet_manager.cc:1918] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:17:15.680045 29577 log.cc:1196] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/9408e3ffed654912bfd977a949561582
I20260809 23:17:15.680315 28683 catalog_manager.cc:4841] TS 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451): tablet 9408e3ffed654912bfd977a949561582 (table test-table [id=d19726722ba84bd194d1ba77be20e06a]) successfully deleted
I20260809 23:17:15.680480 29577 ts_tablet_manager.cc:1939] T 9408e3ffed654912bfd977a949561582 P 968a3fdff4a94c588859f9e93bd115b7: Deleting consensus metadata
I20260809 23:17:15.680958 28685 catalog_manager.cc:4841] TS 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841): tablet 9408e3ffed654912bfd977a949561582 (table test-table [id=d19726722ba84bd194d1ba77be20e06a]) successfully deleted
W20260809 23:17:15.687904 28687 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:40647 (error 108)
I20260809 23:17:15.688470 29529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:15.688345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:40647 (local address 127.0.0.1:48224)
0809 23:17:15.688400 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:15.688436 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 23:17:15.688565 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 29045
W20260809 23:17:15.698182 28687 catalog_manager.cc:4584] TS f1508c984feb44458692d0d59c45257d (127.2.76.193:40647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9408e3ffed654912bfd977a949561582: Network error: Client connection negotiation failed: client connection to 127.2.76.193:40647: connect: Connection refused (error 111)
I20260809 23:17:15.701776 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 29294
I20260809 23:17:15.709297 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 28658
I20260809 23:17:35.718569 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45141
--webserver_interface=127.2.76.254
--webserver_port=37975
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:45141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:17:35.793781 29591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:17:35.793983 29591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:17:35.794018 29591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:17:35.795017 29591 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 23:17:35.795073 29591 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 23:17:35.795092 29591 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 23:17:35.795109 29591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:17:35.795138 29591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:17:35.795161 29591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:17:35.795183 29591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:17:35.795208 29591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:17:35.796721 29591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:45141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:17:35.796985 29591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:17:35.797179 29591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:17:35.805358 29605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:35.805374 29604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:35.805414 29591 server_base.cc:1034] running on GCE node
W20260809 23:17:35.805379 29608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:35.805859 29591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:17:35.806142 29591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:17:35.807304 29591 hybrid_clock.cc:648] HybridClock initialized: now 1786317455807292 us; error 29 us; skew 500 ppm
Aug 09 23:17:35 dist-test-slave-7vjv krb5kdc[28627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317455, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:17:35.816704 29591 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:17:35.817168 29591 webserver.cc:460] Webserver started at http://127.2.76.254:37975/ using document root <none> and password file <none>
I20260809 23:17:35.817386 29591 fs_manager.cc:359] Metadata directory not provided
I20260809 23:17:35.817463 29591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:17:35.818652 29591 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:17:35.819340 29617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:35.819515 29591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:17:35.819629 29591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "053303cba11b488fb38a9823bb381dc5"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "8aa6cab073b70ba113f80d6e9e58d5c5"
server_key_iv: "c98198bc0cdb10ecaaab284fe4ef5dee"
server_key_version: "encryptionkey@0"
I20260809 23:17:35.819972 29591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:17:35.829054 29591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:17:35.830487 29591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:17:35.830636 29591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:17:35.834767 29591 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:45141
I20260809 23:17:35.834796 29707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:45141 every 8 connection(s)
I20260809 23:17:35.835139 29591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:17:35.836449 29709 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:17:35.837049 29709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Bootstrap starting.
I20260809 23:17:35.844221 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 29591
I20260809 23:17:35.844547 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:40647
--local_ip_for_outbound_sockets=127.2.76.193
--tserver_master_addrs=127.2.76.254:45141
--webserver_port=37363
--webserver_interface=127.2.76.193
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:17:35.845737 29709 log.cc:824] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Log is configured to *not* fsync() on all Append() calls
I20260809 23:17:35.847522 29709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:17:35.847808 29709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Bootstrap complete.
I20260809 23:17:35.849943 29709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } }
I20260809 23:17:35.850124 29709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053303cba11b488fb38a9823bb381dc5, State: Initialized, Role: FOLLOWER
I20260809 23:17:35.850262 29709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } }
I20260809 23:17:35.850314 29709 raft_consensus.cc:388] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:17:35.850379 29709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:17:35.850430 29709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 23:17:35.851267 29709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } }
I20260809 23:17:35.851343 29709 leader_election.cc:302] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 053303cba11b488fb38a9823bb381dc5; no voters:
I20260809 23:17:35.851512 29709 leader_election.cc:288] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 23:17:35.851569 29714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 23:17:35.851709 29714 raft_consensus.cc:686] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 2 LEADER]: Becoming Leader. State: Replica: 053303cba11b488fb38a9823bb381dc5, State: Running, Role: LEADER
I20260809 23:17:35.851742 29709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:17:35.851873 29714 consensus_queue.cc:237] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } }
I20260809 23:17:35.852253 29715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "053303cba11b488fb38a9823bb381dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } } }
I20260809 23:17:35.852269 29716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 053303cba11b488fb38a9823bb381dc5. Latest consensus state: current_term: 2 leader_uuid: "053303cba11b488fb38a9823bb381dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } } }
I20260809 23:17:35.852347 29715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [sys.catalog]: This master's current role is: LEADER
I20260809 23:17:35.852365 29716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [sys.catalog]: This master's current role is: LEADER
I20260809 23:17:35.852522 29721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:17:35.853375 29721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:17:35.853535 29721 catalog_manager.cc:1259] Loaded cluster ID: dbd9bbb4f4644c60b3919609d40756d2
I20260809 23:17:35.853781 29721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:17:35.855175 29721 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:17:35.855424 29721 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Loaded TSK: 0
I20260809 23:17:35.867188 29721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Generated new TSK 1
I20260809 23:17:35.867254 29721 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 23:17:35.922590 29713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:17:35.922776 29713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:17:35.922811 29713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:17:35.923810 29713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:17:35.923869 29713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:17:35.923908 29713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:17:35.925203 29713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:40647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=37363
--tserver_master_addrs=127.2.76.254:45141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:17:35.925428 29713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:17:35.925630 29713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:17:35.929908 29760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:35.929935 29756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:35.930145 29752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:35.930104 29713 server_base.cc:1034] running on GCE node
I20260809 23:17:35.930320 29713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:17:35.930572 29713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:17:35.931758 29713 hybrid_clock.cc:648] HybridClock initialized: now 1786317455931746 us; error 39 us; skew 500 ppm
Aug 09 23:17:35 dist-test-slave-7vjv krb5kdc[28627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317455, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:17:35.934655 29713 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:17:35.935107 29713 webserver.cc:460] Webserver started at http://127.2.76.193:37363/ using document root <none> and password file <none>
I20260809 23:17:35.935320 29713 fs_manager.cc:359] Metadata directory not provided
I20260809 23:17:35.935395 29713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:17:35.936694 29713 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 23:17:35.937390 29769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:35.937551 29713 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:17:35.937642 29713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "f1508c984feb44458692d0d59c45257d"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "38a5bd66fafb0b68c4705e020b5399e8"
server_key_iv: "f5e256cf8b56d8ef11e98d5eea0e8c0d"
server_key_version: "encryptionkey@0"
I20260809 23:17:35.937928 29713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:17:35.950958 29713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:17:35.952934 29713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:17:35.953083 29713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:17:35.953321 29713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:17:35.953696 29784 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 23:17:35.955137 29713 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 23:17:35.955189 29713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 23:17:35.955224 29713 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 23:17:35.955757 29713 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 23:17:35.955809 29713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 23:17:35.955855 29784 tablet_bootstrap.cc:492] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: Bootstrap starting.
I20260809 23:17:35.962242 29713 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:40647
I20260809 23:17:35.962555 29713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:17:35.962584 29969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:40647 every 8 connection(s)
I20260809 23:17:35.965863 29784 log.cc:824] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:17:35.967871 29784 tablet_bootstrap.cc:492] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:17:35.968232 29784 tablet_bootstrap.cc:492] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: Bootstrap complete.
I20260809 23:17:35.969134 29784 ts_tablet_manager.cc:1397] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.010s
Aug 09 23:17:35 dist-test-slave-7vjv krb5kdc[28627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317455, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:17:35.970227 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 29713
I20260809 23:17:35.970108 29784 raft_consensus.cc:348] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } } peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } }
I20260809 23:17:35.970410 29784 raft_consensus.cc:729] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1508c984feb44458692d0d59c45257d, State: Initialized, Role: FOLLOWER
I20260809 23:17:35.970510 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:43841
--local_ip_for_outbound_sockets=127.2.76.194
--tserver_master_addrs=127.2.76.254:45141
--webserver_port=41207
--webserver_interface=127.2.76.194
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:17:35.970544 29784 consensus_queue.cc:260] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } } peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } }
I20260809 23:17:35.970832 29784 ts_tablet_manager.cc:1428] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 23:17:35.973426 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:35.963286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:34831 (local address 127.2.76.254:45141)
0809 23:17:35.963457 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:35.963465 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:35.963532 (+ 67us) server_negotiation.cc:407] Connection header received
0809 23:17:35.964350 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:35.964355 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:35.964393 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:35.964502 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:17:35.965344 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:35.966124 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:35.967039 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:35.967562 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:35.967665 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:35.969712 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:17:35.969715 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:17:35.969715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:17:35.969723 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:17:35.971568 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:35.971956 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:35.971959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:35.971959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:35.972073 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:35.972612 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:35.972612 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:35.972613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:35.972802 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:17:35.972866 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:35.973195 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 23:17:35.973297 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260809 23:17:35.973516 29973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:35.963158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:34831)
0809 23:17:35.963360 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:35.963439 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:35.964145 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:35.964584 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:35.964589 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:35.964955 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:17:35.965233 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:35.965239 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:35.966231 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:35.966233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:35.966944 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:35.966947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:35.967660 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:35.967661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:35.967748 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:35.968234 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:17:35.968241 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:17:35.969602 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:17:35.971656 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:35.971661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:35.971662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:35.971872 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:35.972398 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:35.972399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:35.972400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:35.972538 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:35.972873 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:17:35.972878 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:17:35.973038 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 23:17:35.973148 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 23:17:35.973318 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":104,"threads_started":1}
I20260809 23:17:35.974123 29970 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45141
I20260809 23:17:35.974220 29970 heartbeater.cc:464] Registering TS with master...
I20260809 23:17:35.974366 29970 heartbeater.cc:511] Master 127.2.76.254:45141 requested a full tablet report, sending...
I20260809 23:17:35.975010 29646 ts_manager.cc:194] Registered new tserver with Master: f1508c984feb44458692d0d59c45257d (127.2.76.193:40647)
I20260809 23:17:35.975994 29646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:34831
I20260809 23:17:35.976744 29970 heartbeater.cc:503] Master 127.2.76.254:45141 was elected leader, sending a full tablet report...
I20260809 23:17:35.982730 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:35.975560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:40647 (local address 127.0.0.1:46292)
0809 23:17:35.980182 (+ 4622us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:35.980207 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:35.980284 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:35.980455 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:35.980456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:35.980458 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:17:35.980521 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:35.980523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:35.981218 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:35.981219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:35.981894 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:35.981896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:35.982449 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:35.982450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:35.982533 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:35.982553 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 23:17:35.982625 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 23:17:35.982673 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4600,"thread_start_us":75,"threads_started":1}
I20260809 23:17:35.982729 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:35.975635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46292 (local address 127.2.76.193:40647)
0809 23:17:35.975772 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:35.975775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:35.980270 (+ 4495us) server_negotiation.cc:407] Connection header received
0809 23:17:35.980370 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:35.980372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:35.980385 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:35.980451 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:17:35.980584 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:35.981138 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:35.981976 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:35.982368 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:35.982447 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:35.982496 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:35.982639 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 23:17:35.982673 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260809 23:17:35.983117 29853 tablet_service.cc:1514] Processing DeleteTablet for tablet 9408e3ffed654912bfd977a949561582 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:17:15 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:46292
I20260809 23:17:35.983518 29987 tablet_replica.cc:330] stopping tablet replica
I20260809 23:17:35.983613 29987 raft_consensus.cc:2229] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:17:35.983678 29987 raft_consensus.cc:2258] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:17:35.983938 29987 ts_tablet_manager.cc:1905] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:17:35.986038 29987 ts_tablet_manager.cc:1918] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:17:35.986120 29987 log.cc:1196] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal/wals/9408e3ffed654912bfd977a949561582
I20260809 23:17:35.986383 29987 ts_tablet_manager.cc:1939] T 9408e3ffed654912bfd977a949561582 P f1508c984feb44458692d0d59c45257d: Deleting consensus metadata
I20260809 23:17:35.986828 29622 catalog_manager.cc:4841] TS f1508c984feb44458692d0d59c45257d (127.2.76.193:40647): tablet 9408e3ffed654912bfd977a949561582 (table test-table [id=d19726722ba84bd194d1ba77be20e06a]) successfully deleted
W20260809 23:17:36.046240 29981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:17:36.046394 29981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:17:36.046414 29981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:17:36.047518 29981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:17:36.047559 29981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:17:36.047585 29981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:17:36.048919 29981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:43841
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=41207
--tserver_master_addrs=127.2.76.254:45141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:17:36.049117 29981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:17:36.049312 29981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:17:36.053606 30003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:36.053689 29994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:36.053648 29995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:36.053768 29981 server_base.cc:1034] running on GCE node
I20260809 23:17:36.053959 29981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:17:36.054183 29981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:17:36.055320 29981 hybrid_clock.cc:648] HybridClock initialized: now 1786317456055286 us; error 49 us; skew 500 ppm
Aug 09 23:17:36 dist-test-slave-7vjv krb5kdc[28627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317456, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:17:36.057480 29981 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:17:36.057879 29981 webserver.cc:460] Webserver started at http://127.2.76.194:41207/ using document root <none> and password file <none>
I20260809 23:17:36.058025 29981 fs_manager.cc:359] Metadata directory not provided
I20260809 23:17:36.058068 29981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:17:36.059149 29981 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:17:36.059626 30020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:36.059788 29981 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:36.059917 29981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "968a3fdff4a94c588859f9e93bd115b7"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "ac4dafe8a1d877298a20bb3399f674ca"
server_key_iv: "0f3388828e110b2c28f773b99bb2df9c"
server_key_version: "encryptionkey@0"
I20260809 23:17:36.060251 29981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:17:36.073307 29981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:17:36.074817 29981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:17:36.074961 29981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:17:36.075155 29981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:17:36.075423 29981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:17:36.075470 29981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:36.075505 29981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:17:36.075527 29981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:36.081123 29981 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:43841
I20260809 23:17:36.081161 30198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:43841 every 8 connection(s)
I20260809 23:17:36.081477 29981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:17:36 dist-test-slave-7vjv krb5kdc[28627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317456, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:17:36.087134 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 29981
I20260809 23:17:36.087359 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:39451
--local_ip_for_outbound_sockets=127.2.76.195
--tserver_master_addrs=127.2.76.254:45141
--webserver_port=44641
--webserver_interface=127.2.76.195
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:17:36.089869 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.082121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:46521 (local address 127.2.76.254:45141)
0809 23:17:36.082160 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:36.082162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:36.082363 (+ 201us) server_negotiation.cc:407] Connection header received
0809 23:17:36.082788 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:36.082789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:36.082804 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:36.082848 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:17:36.083605 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.084068 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.085057 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.085284 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.085371 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.087331 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:17:36.087332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:17:36.087333 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:17:36.087340 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:17:36.088666 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:36.089022 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:36.089023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:36.089023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:36.089079 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:36.089298 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:36.089299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:36.089299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:36.089408 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:17:36.089473 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:36.089706 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 23:17:36.089790 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:17:36.090023 30202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.082095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:46521)
0809 23:17:36.082256 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:36.082285 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:36.082690 (+ 405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:36.082905 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:36.082908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:36.083368 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:17:36.083517 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:36.083521 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.084202 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.084203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:36.084966 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:36.084969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.085383 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.085384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:36.085510 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.085945 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:17:36.085952 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:17:36.087191 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:17:36.088745 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:36.088752 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:36.088754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:36.088951 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:36.089140 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:36.089141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:36.089141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:36.089239 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:36.089476 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:17:36.089483 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:17:36.089597 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 23:17:36.089686 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 23:17:36.089814 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260809 23:17:36.090452 30199 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45141
I20260809 23:17:36.090544 30199 heartbeater.cc:464] Registering TS with master...
I20260809 23:17:36.090690 30199 heartbeater.cc:511] Master 127.2.76.254:45141 requested a full tablet report, sending...
I20260809 23:17:36.091027 29646 ts_manager.cc:194] Registered new tserver with Master: 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841)
I20260809 23:17:36.091468 29646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:46521
W20260809 23:17:36.160207 30204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:17:36.160368 30204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:17:36.160389 30204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:17:36.161305 30204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:17:36.161339 30204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:17:36.161363 30204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:17:36.162649 30204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:39451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=44641
--tserver_master_addrs=127.2.76.254:45141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:17:36.162801 30204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:17:36.162951 30204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:17:36.167361 30204 server_base.cc:1034] running on GCE node
W20260809 23:17:36.167336 30214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:36.167323 30209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:36.167335 30210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:36.167758 30204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:17:36.167930 30204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:17:36.169056 30204 hybrid_clock.cc:648] HybridClock initialized: now 1786317456169031 us; error 38 us; skew 500 ppm
Aug 09 23:17:36 dist-test-slave-7vjv krb5kdc[28627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317456, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:17:36.171394 30204 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:17:36.171711 30204 webserver.cc:460] Webserver started at http://127.2.76.195:44641/ using document root <none> and password file <none>
I20260809 23:17:36.171866 30204 fs_manager.cc:359] Metadata directory not provided
I20260809 23:17:36.171913 30204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:17:36.173115 30204 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:17:36.173612 30226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:36.173763 30204 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:36.173864 30204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "108c8e78f0c64c3b9295f7f9991232ba"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "013e0fe73180527fe6504179c3ebe2af"
server_key_iv: "4e514ac7c3b6d484a680918153a0a5a2"
server_key_version: "encryptionkey@0"
I20260809 23:17:36.174134 30204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:17:36.205641 30204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:17:36.207245 30204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:17:36.207382 30204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:17:36.207572 30204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:17:36.207820 30204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:17:36.207875 30204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:36.207911 30204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:17:36.207932 30204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:36.213542 30204 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:39451
I20260809 23:17:36.213577 30416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:39451 every 8 connection(s)
I20260809 23:17:36.213891 30204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:17:36 dist-test-slave-7vjv krb5kdc[28627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317456, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:17:36.221676 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.214548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:40927 (local address 127.2.76.254:45141)
0809 23:17:36.214617 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:36.214620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:36.214830 (+ 210us) server_negotiation.cc:407] Connection header received
0809 23:17:36.215217 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:36.215218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:36.215227 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:36.215284 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:17:36.216000 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.216476 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.217278 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.217448 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.217492 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.219372 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:17:36.219374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:17:36.219374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:17:36.219381 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:17:36.220471 (+ 1090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:36.220814 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:36.220815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:36.220815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:36.220868 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:36.221094 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:36.221095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:36.221095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:36.221190 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:17:36.221255 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:36.221515 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 23:17:36.221595 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:17:36.221769 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.214528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:40927)
0809 23:17:36.214718 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:36.214749 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:36.215144 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:36.215325 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:36.215328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:36.215789 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:17:36.215935 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:36.215939 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.216543 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.216544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:36.217203 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:36.217205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.217527 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.217528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:36.217612 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.218086 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:17:36.218093 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:17:36.219264 (+ 1171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:17:36.220546 (+ 1282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:36.220553 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:36.220555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:36.220743 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:36.220935 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:36.220935 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:36.220936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:36.221027 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:36.221248 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:17:36.221255 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:17:36.221371 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 23:17:36.221464 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 23:17:36.221595 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":116,"threads_started":1}
I20260809 23:17:36.222213 30417 heartbeater.cc:347] Connected to a master server at 127.2.76.254:45141
I20260809 23:17:36.222286 30417 heartbeater.cc:464] Registering TS with master...
I20260809 23:17:36.222447 30417 heartbeater.cc:511] Master 127.2.76.254:45141 requested a full tablet report, sending...
I20260809 23:17:36.222795 29646 ts_manager.cc:194] Registered new tserver with Master: 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451)
I20260809 23:17:36.223131 29646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:40927
I20260809 23:17:36.223446 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 30204
I20260809 23:17:36.228960 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.223834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59630 (local address 127.2.76.254:45141)
0809 23:17:36.223871 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:36.223873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:36.224035 (+ 162us) server_negotiation.cc:407] Connection header received
0809 23:17:36.224137 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:36.224137 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:36.224146 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:36.224207 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:17:36.224723 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.225119 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.225760 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.225909 (+ 149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.225952 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.227018 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:17:36.227018 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:17:36.227018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:17:36.227023 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:17:36.228039 (+ 1016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:36.228388 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:36.228389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:36.228389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:36.228436 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:36.228590 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:36.228590 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:36.228590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:36.228667 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:17:36.228773 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:36.228831 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 23:17:36.228884 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:17:36.229614 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:17:36.232008 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.229937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59636 (local address 127.2.76.254:45141)
0809 23:17:36.229982 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:36.229984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:36.230077 (+ 93us) server_negotiation.cc:407] Connection header received
0809 23:17:36.230162 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:36.230163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:36.230188 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:36.230261 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:17:36.230344 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.230740 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.231406 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.231572 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.231632 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.231786 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:17:36.231820 (+ 34us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 23:17:36.231954 (+ 134us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59636: token expired
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 23:17:36.232055 29975 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59636: token expired
W20260809 23:17:36.232216 29520 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.76.254:45141)
I20260809 23:17:36.237079 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.232451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59650 (local address 127.2.76.254:45141)
0809 23:17:36.232482 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:36.232484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:36.232492 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:17:36.232519 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:36.232519 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:36.232526 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:36.232562 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:17:36.232994 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.233403 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.233993 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.234186 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.234252 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.235155 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:17:36.235156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:17:36.235157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:17:36.235163 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:17:36.236084 (+ 921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:36.236400 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:36.236401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:36.236401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:36.236447 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:36.236623 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:36.236624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:36.236624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:36.236735 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:17:36.236822 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:36.236942 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 23:17:36.237003 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 23:17:36.237967 29646 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:17:36.239279 29646 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:17:36.241364 29853 tablet_service.cc:1467] Processing CreateTablet for tablet 2e5afb46a6eb4187b7580afd3cfb8fa6 (DEFAULT_TABLE table=test-table [id=dfe3c06ffcc34c42be7ffd25520642f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:17:36.241551 29853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e5afb46a6eb4187b7580afd3cfb8fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:17:36.242985 29784 tablet_bootstrap.cc:492] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d: Bootstrap starting.
I20260809 23:17:36.243312 29784 tablet_bootstrap.cc:654] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d: Neither blocks nor log segments found. Creating new log.
I20260809 23:17:36.243819 29784 tablet_bootstrap.cc:492] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d: No bootstrap required, opened a new log
I20260809 23:17:36.243889 29784 ts_tablet_manager.cc:1397] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 23:17:36.244038 29784 raft_consensus.cc:348] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:17:36.244117 29784 raft_consensus.cc:374] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:17:36.244163 29784 raft_consensus.cc:729] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1508c984feb44458692d0d59c45257d, State: Initialized, Role: FOLLOWER
I20260809 23:17:36.244207 29784 consensus_queue.cc:260] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:17:36.244305 29784 ts_tablet_manager.cc:1428] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:36.244585 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.241120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:39451 (local address 127.0.0.1:48006)
0809 23:17:36.241162 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:36.241171 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:36.241205 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:36.241521 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:36.241522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:36.241523 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:17:36.241556 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:36.241557 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.242469 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.242471 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:36.243379 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:36.243383 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.244383 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.244384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:36.244438 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.244441 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:17:36.244470 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 23:17:36.244527 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 23:17:36.244865 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.241173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48006 (local address 127.2.76.195:39451)
0809 23:17:36.241343 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:36.241346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:36.241355 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:17:36.241379 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:36.241380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:36.241389 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:36.241511 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:17:36.241622 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.242396 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.243520 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.244239 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.244677 (+ 438us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.244739 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:36.244763 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 23:17:36.244795 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":59,"threads_started":1}
I20260809 23:17:36.245463 30303 tablet_service.cc:1467] Processing CreateTablet for tablet 2e5afb46a6eb4187b7580afd3cfb8fa6 (DEFAULT_TABLE table=test-table [id=dfe3c06ffcc34c42be7ffd25520642f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:17:36.245775 30303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e5afb46a6eb4187b7580afd3cfb8fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:17:36.246531 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.241137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:43841 (local address 127.0.0.1:47516)
0809 23:17:36.243022 (+ 1885us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:36.243032 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:36.243097 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:36.243290 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:36.243291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:36.243294 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:17:36.243336 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:36.243337 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.244523 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.244524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:36.245473 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:36.245476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.246280 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.246280 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:36.246338 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.246341 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:17:36.246399 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 23:17:36.246457 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1873,"thread_start_us":55,"threads_started":1}
I20260809 23:17:36.246543 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.241320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47516 (local address 127.2.76.194:43841)
0809 23:17:36.241503 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:36.241506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:36.243146 (+ 1640us) server_negotiation.cc:407] Connection header received
0809 23:17:36.243189 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:36.243191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:36.243205 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:36.243269 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:17:36.243412 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.244442 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.245569 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.246204 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.246275 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.246323 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:36.246424 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 23:17:36.246459 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260809 23:17:36.247175 30099 tablet_service.cc:1467] Processing CreateTablet for tablet 2e5afb46a6eb4187b7580afd3cfb8fa6 (DEFAULT_TABLE table=test-table [id=dfe3c06ffcc34c42be7ffd25520642f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:17:36.247481 30099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e5afb46a6eb4187b7580afd3cfb8fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:17:36.248762 30428 tablet_bootstrap.cc:492] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba: Bootstrap starting.
I20260809 23:17:36.249279 30428 tablet_bootstrap.cc:654] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba: Neither blocks nor log segments found. Creating new log.
I20260809 23:17:36.249711 30428 log.cc:824] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba: Log is configured to *not* fsync() on all Append() calls
I20260809 23:17:36.250079 30431 tablet_bootstrap.cc:492] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7: Bootstrap starting.
I20260809 23:17:36.250561 30428 tablet_bootstrap.cc:492] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba: No bootstrap required, opened a new log
I20260809 23:17:36.250633 30428 ts_tablet_manager.cc:1397] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:17:36.250833 30431 tablet_bootstrap.cc:654] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7: Neither blocks nor log segments found. Creating new log.
I20260809 23:17:36.251369 30431 log.cc:824] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7: Log is configured to *not* fsync() on all Append() calls
I20260809 23:17:36.252152 30431 tablet_bootstrap.cc:492] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7: No bootstrap required, opened a new log
I20260809 23:17:36.252219 30431 ts_tablet_manager.cc:1397] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:17:36.252564 30428 raft_consensus.cc:348] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:17:36.252729 30428 raft_consensus.cc:374] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:17:36.252758 30428 raft_consensus.cc:729] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 108c8e78f0c64c3b9295f7f9991232ba, State: Initialized, Role: FOLLOWER
I20260809 23:17:36.252872 30428 consensus_queue.cc:260] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:17:36.253147 30428 ts_tablet_manager.cc:1428] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:17:36.253261 30417 heartbeater.cc:503] Master 127.2.76.254:45141 was elected leader, sending a full tablet report...
I20260809 23:17:36.254359 30431 raft_consensus.cc:348] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:17:36.254485 30431 raft_consensus.cc:374] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:17:36.254515 30431 raft_consensus.cc:729] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 968a3fdff4a94c588859f9e93bd115b7, State: Initialized, Role: FOLLOWER
I20260809 23:17:36.254586 30431 consensus_queue.cc:260] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:17:36.254781 30431 ts_tablet_manager.cc:1428] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 23:17:36.254837 30199 heartbeater.cc:503] Master 127.2.76.254:45141 was elected leader, sending a full tablet report...
I20260809 23:17:36.304316 30433 raft_consensus.cc:482] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:17:36.304399 30433 raft_consensus.cc:504] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:17:36.304651 30433 leader_election.cc:288] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841), f1508c984feb44458692d0d59c45257d (127.2.76.193:40647)
I20260809 23:17:36.307956 30420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.304765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:43841 (local address 127.2.76.195:58067)
0809 23:17:36.304838 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:36.304864 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:36.304903 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:36.305070 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:36.305071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:36.305072 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:17:36.305140 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:36.305142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.305915 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.305916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:36.307013 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:36.307016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.307667 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.307668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:36.307765 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.307766 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 23:17:36.307827 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 23:17:36.307894 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":58}
I20260809 23:17:36.307978 29986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.304840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:35181 (local address 127.2.76.193:40647)
0809 23:17:36.304888 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:36.304890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:36.305024 (+ 134us) server_negotiation.cc:407] Connection header received
0809 23:17:36.305051 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:36.305052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:36.305062 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:36.305112 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:17:36.305231 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.305841 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.307100 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.307590 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.307644 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.307650 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:36.307864 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 23:17:36.307921 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 23:17:36.307967 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.304785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:40647 (local address 127.2.76.195:35181)
0809 23:17:36.304947 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:36.304956 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:36.305003 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:36.305113 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:36.305114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:36.305115 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:17:36.305168 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:36.305169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.305915 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.305916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:36.307013 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:36.307016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.307667 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.307668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:36.307761 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.307764 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:17:36.307827 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 23:17:36.307863 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":76,"thread_start_us":89,"threads_started":1}
I20260809 23:17:36.307978 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.304865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:58067 (local address 127.2.76.194:43841)
0809 23:17:36.304904 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:36.304906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:36.304914 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:17:36.304955 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:36.304956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:36.304961 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:36.305045 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:17:36.305227 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.305837 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.307100 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.307572 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.307643 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.307650 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:36.307864 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 23:17:36.307919 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:17:36.308302 30130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e5afb46a6eb4187b7580afd3cfb8fa6" candidate_uuid: "108c8e78f0c64c3b9295f7f9991232ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "968a3fdff4a94c588859f9e93bd115b7" is_pre_election: true
I20260809 23:17:36.308341 29882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e5afb46a6eb4187b7580afd3cfb8fa6" candidate_uuid: "108c8e78f0c64c3b9295f7f9991232ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1508c984feb44458692d0d59c45257d" is_pre_election: true
I20260809 23:17:36.308436 30130 raft_consensus.cc:2454] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 108c8e78f0c64c3b9295f7f9991232ba in term 0.
I20260809 23:17:36.308463 29882 raft_consensus.cc:2454] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 108c8e78f0c64c3b9295f7f9991232ba in term 0.
I20260809 23:17:36.308646 30237 leader_election.cc:302] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 108c8e78f0c64c3b9295f7f9991232ba, f1508c984feb44458692d0d59c45257d; no voters:
I20260809 23:17:36.308796 30433 raft_consensus.cc:2781] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:17:36.308876 30433 raft_consensus.cc:482] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:17:36.308907 30433 raft_consensus.cc:3037] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:17:36.309473 30433 raft_consensus.cc:504] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:17:36.309608 30433 leader_election.cc:288] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [CANDIDATE]: Term 1 election: Requested vote from peers 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841), f1508c984feb44458692d0d59c45257d (127.2.76.193:40647)
I20260809 23:17:36.309743 29882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e5afb46a6eb4187b7580afd3cfb8fa6" candidate_uuid: "108c8e78f0c64c3b9295f7f9991232ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1508c984feb44458692d0d59c45257d"
I20260809 23:17:36.309767 30130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e5afb46a6eb4187b7580afd3cfb8fa6" candidate_uuid: "108c8e78f0c64c3b9295f7f9991232ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "968a3fdff4a94c588859f9e93bd115b7"
I20260809 23:17:36.309820 30130 raft_consensus.cc:3037] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:17:36.309818 29882 raft_consensus.cc:3037] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:17:36.310647 30130 raft_consensus.cc:2454] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 108c8e78f0c64c3b9295f7f9991232ba in term 1.
I20260809 23:17:36.310876 30230 leader_election.cc:302] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 108c8e78f0c64c3b9295f7f9991232ba, 968a3fdff4a94c588859f9e93bd115b7; no voters:
I20260809 23:17:36.310976 30433 raft_consensus.cc:2781] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:17:36.310990 29882 raft_consensus.cc:2454] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 108c8e78f0c64c3b9295f7f9991232ba in term 1.
I20260809 23:17:36.311151 30433 raft_consensus.cc:686] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [term 1 LEADER]: Becoming Leader. State: Replica: 108c8e78f0c64c3b9295f7f9991232ba, State: Running, Role: LEADER
I20260809 23:17:36.311240 30433 consensus_queue.cc:237] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:17:36.311975 29646 catalog_manager.cc:5495] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195). New cstate: current_term: 1 leader_uuid: "108c8e78f0c64c3b9295f7f9991232ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } health_report { overall_health: UNKNOWN } } }
I20260809 23:17:36.320554 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.318694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48018 (local address 127.2.76.195:39451)
0809 23:17:36.318744 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:36.318746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:36.318753 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:17:36.318784 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:36.318784 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:36.318795 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:36.318845 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:17:36.318980 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.319342 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.319942 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:36.320164 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:36.320217 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:36.320326 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:17:36.320397 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:17:36.320451 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:36.320485 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 23:17:36.320500 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 23:17:36.321939 29882 raft_consensus.cc:1261] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Refusing update from remote peer 108c8e78f0c64c3b9295f7f9991232ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:17:36.322201 30130 raft_consensus.cc:1261] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7 [term 1 FOLLOWER]: Refusing update from remote peer 108c8e78f0c64c3b9295f7f9991232ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:17:36.322232 30433 consensus_queue.cc:1035] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:17:36.322449 30433 consensus_queue.cc:1035] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:17:36.324983 30444 mvcc.cc:205] Tried to move back new op lower bound from 7316756301092462592 to 7316756301051461632. Current Snapshot: MvccSnapshot[applied={T|T < 7316756301092462592}]
I20260809 23:17:36.325275 30443 mvcc.cc:205] Tried to move back new op lower bound from 7316756301092462592 to 7316756301051461632. Current Snapshot: MvccSnapshot[applied={T|T < 7316756301092462592 or (T in {7316756301092462592})}]
I20260809 23:17:36.326752 29643 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:17:36.326858 29643 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:17:36.327425 29643 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Sending DeleteTablet for 3 replicas of tablet 2e5afb46a6eb4187b7580afd3cfb8fa6
I20260809 23:17:36.327706 29853 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e5afb46a6eb4187b7580afd3cfb8fa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:17:36 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:46292
I20260809 23:17:36.327715 30303 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e5afb46a6eb4187b7580afd3cfb8fa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:17:36 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:48006
I20260809 23:17:36.327800 29987 tablet_replica.cc:330] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d: stopping tablet replica
I20260809 23:17:36.327850 29987 raft_consensus.cc:2229] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:17:36.327901 29987 raft_consensus.cc:2258] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:17:36.327904 30456 tablet_replica.cc:330] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba: stopping tablet replica
I20260809 23:17:36.327915 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 29713
I20260809 23:17:36.327983 30456 raft_consensus.cc:2229] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:17:36.328053 30456 raft_consensus.cc:2258] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:17:36.328485 30456 ts_tablet_manager.cc:1905] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:17:36.330022 30456 ts_tablet_manager.cc:1918] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:17:36.330153 30456 log.cc:1196] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/2e5afb46a6eb4187b7580afd3cfb8fa6
I20260809 23:17:36.330523 30456 ts_tablet_manager.cc:1939] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 108c8e78f0c64c3b9295f7f9991232ba: Deleting consensus metadata
I20260809 23:17:36.331009 29618 catalog_manager.cc:4841] TS 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451): tablet 2e5afb46a6eb4187b7580afd3cfb8fa6 (table test-table [id=dfe3c06ffcc34c42be7ffd25520642f7]) successfully deleted
I20260809 23:17:36.350222 30099 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e5afb46a6eb4187b7580afd3cfb8fa6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:17:36 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:47516
I20260809 23:17:36.351059 30458 tablet_replica.cc:330] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7: stopping tablet replica
I20260809 23:17:36.351229 30458 raft_consensus.cc:2229] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:17:36.351409 30458 raft_consensus.cc:2258] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:17:36.353049 30458 ts_tablet_manager.cc:1905] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:17:36.354773 30458 ts_tablet_manager.cc:1918] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:17:36.355281 30458 log.cc:1196] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/2e5afb46a6eb4187b7580afd3cfb8fa6
I20260809 23:17:36.355600 30458 ts_tablet_manager.cc:1939] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P 968a3fdff4a94c588859f9e93bd115b7: Deleting consensus metadata
I20260809 23:17:36.356033 29619 catalog_manager.cc:4841] TS 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841): tablet 2e5afb46a6eb4187b7580afd3cfb8fa6 (table test-table [id=dfe3c06ffcc34c42be7ffd25520642f7]) successfully deleted
W20260809 23:17:36.360576 29622 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:40647 (error 108)
I20260809 23:17:36.361011 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 29981
I20260809 23:17:36.361081 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:36.360939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:40647 (local address 127.0.0.1:46296)
0809 23:17:36.361003 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:36.361048 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:40647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 23:17:36.365350 29622 catalog_manager.cc:4584] TS f1508c984feb44458692d0d59c45257d (127.2.76.193:40647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e5afb46a6eb4187b7580afd3cfb8fa6: Network error: Client connection negotiation failed: client connection to 127.2.76.193:40647: connect: Connection refused (error 111)
I20260809 23:17:36.369671 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 30204
I20260809 23:17:36.380437 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 29591
I20260809 23:17:56.392549 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45141
--webserver_interface=127.2.76.254
--webserver_port=37975
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:45141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:17:56.467288 30461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:17:56.467491 30461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:17:56.467525 30461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:17:56.468631 30461 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 23:17:56.468688 30461 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 23:17:56.468708 30461 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 23:17:56.468725 30461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:17:56.468755 30461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:17:56.468788 30461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:17:56.468811 30461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:17:56.468837 30461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:17:56.470198 30461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:45141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:45141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=37975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:17:56.470434 30461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:17:56.470629 30461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:17:56.475329 30470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:56.475286 30471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:56.475391 30473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:56.475334 30461 server_base.cc:1034] running on GCE node
I20260809 23:17:56.475768 30461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:17:56.476042 30461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:17:56.477217 30461 hybrid_clock.cc:648] HybridClock initialized: now 1786317476477202 us; error 35 us; skew 500 ppm
Aug 09 23:17:56 dist-test-slave-7vjv krb5kdc[28627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317476, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:17:56.482594 30461 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:17:56.482966 30461 webserver.cc:460] Webserver started at http://127.2.76.254:37975/ using document root <none> and password file <none>
I20260809 23:17:56.483171 30461 fs_manager.cc:359] Metadata directory not provided
I20260809 23:17:56.483233 30461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:17:56.484508 30461 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:17:56.485123 30494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:56.485278 30461 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:17:56.485371 30461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "053303cba11b488fb38a9823bb381dc5"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "8aa6cab073b70ba113f80d6e9e58d5c5"
server_key_iv: "c98198bc0cdb10ecaaab284fe4ef5dee"
server_key_version: "encryptionkey@0"
I20260809 23:17:56.485675 30461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:17:56.506669 30461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:17:56.508263 30461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:17:56.508406 30461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:17:56.512027 30461 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:45141
I20260809 23:17:56.512077 30586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:45141 every 8 connection(s)
I20260809 23:17:56.512446 30461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:17:56.513712 30587 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:17:56.514247 30587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Bootstrap starting.
I20260809 23:17:56.518435 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 30461
I20260809 23:17:56.518679 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:40647
--local_ip_for_outbound_sockets=127.2.76.193
--tserver_master_addrs=127.2.76.254:45141
--webserver_port=37363
--webserver_interface=127.2.76.193
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:17:56.523347 30587 log.cc:824] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Log is configured to *not* fsync() on all Append() calls
I20260809 23:17:56.525352 30587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:17:56.525630 30587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Bootstrap complete.
I20260809 23:17:56.527609 30587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } }
I20260809 23:17:56.527819 30587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053303cba11b488fb38a9823bb381dc5, State: Initialized, Role: FOLLOWER
I20260809 23:17:56.528013 30587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } }
I20260809 23:17:56.528090 30587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:17:56.528154 30587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:17:56.528200 30587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 23:17:56.529083 30587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } }
I20260809 23:17:56.529197 30587 leader_election.cc:302] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 053303cba11b488fb38a9823bb381dc5; no voters:
I20260809 23:17:56.529377 30587 leader_election.cc:288] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 23:17:56.529412 30591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 23:17:56.529531 30591 raft_consensus.cc:686] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [term 3 LEADER]: Becoming Leader. State: Replica: 053303cba11b488fb38a9823bb381dc5, State: Running, Role: LEADER
I20260809 23:17:56.529630 30587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:17:56.529671 30591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } }
I20260809 23:17:56.529947 30591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 053303cba11b488fb38a9823bb381dc5. Latest consensus state: current_term: 3 leader_uuid: "053303cba11b488fb38a9823bb381dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } } }
I20260809 23:17:56.529948 30592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "053303cba11b488fb38a9823bb381dc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "053303cba11b488fb38a9823bb381dc5" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 45141 } } }
I20260809 23:17:56.530197 30592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [sys.catalog]: This master's current role is: LEADER
I20260809 23:17:56.530130 30591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5 [sys.catalog]: This master's current role is: LEADER
I20260809 23:17:56.530375 30603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:17:56.531177 30603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:17:56.531334 30603 catalog_manager.cc:1259] Loaded cluster ID: dbd9bbb4f4644c60b3919609d40756d2
I20260809 23:17:56.531582 30603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:17:56.532344 30603 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:17:56.532521 30603 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Loaded TSK: 0
I20260809 23:17:56.532572 30603 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 053303cba11b488fb38a9823bb381dc5: Loaded TSK: 1
F20260809 23:17:56.532770 30603 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f35c8ba19ad google::LogMessage::Fail() at ??:0
@ 0x7f35c8ba488a google::LogMessage::SendToLog() at ??:0
@ 0x7f35c8ba14de google::LogMessage::Flush() at ??:0
@ 0x7f35c8ba2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f35cb30e104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f35cb30bd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f35d0afb1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f35d0afb48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f35d0afb594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f35d0ac1c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f35d0acbad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f35caa6f0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f35caa64954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f35cb8c86db start_thread at ??:0
@ 0x7f35c769f61f clone at ??:0
W20260809 23:17:56.614285 30589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:17:56.614440 30589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:17:56.614459 30589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:17:56.615401 30589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:17:56.615437 30589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:17:56.615458 30589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:17:56.616808 30589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:40647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=37363
--tserver_master_addrs=127.2.76.254:45141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:17:56.617014 30589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:17:56.617218 30589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:17:56.621510 30652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:56.621531 30649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:56.621654 30589 server_base.cc:1034] running on GCE node
W20260809 23:17:56.621779 30656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:56.621989 30589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:17:56.622233 30589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:17:56.623406 30589 hybrid_clock.cc:648] HybridClock initialized: now 1786317476623394 us; error 25 us; skew 500 ppm
Aug 09 23:17:56 dist-test-slave-7vjv krb5kdc[28627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317476, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:17:56.626104 30589 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:17:56.626622 30589 webserver.cc:460] Webserver started at http://127.2.76.193:37363/ using document root <none> and password file <none>
I20260809 23:17:56.626842 30589 fs_manager.cc:359] Metadata directory not provided
I20260809 23:17:56.626905 30589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:17:56.628368 30589 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:17:56.629060 30666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:56.629208 30589 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:56.629280 30589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "f1508c984feb44458692d0d59c45257d"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "38a5bd66fafb0b68c4705e020b5399e8"
server_key_iv: "f5e256cf8b56d8ef11e98d5eea0e8c0d"
server_key_version: "encryptionkey@0"
I20260809 23:17:56.629611 30589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:17:56.650051 30589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:17:56.651962 30589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:17:56.652082 30589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:17:56.652318 30589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:17:56.652694 30683 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 23:17:56.654265 30589 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 23:17:56.654322 30589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 23:17:56.654353 30589 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 23:17:56.654882 30589 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 23:17:56.654976 30589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 23:17:56.655076 30683 tablet_bootstrap.cc:492] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d: Bootstrap starting.
I20260809 23:17:56.661957 30589 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:40647
I20260809 23:17:56.662014 30849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:40647 every 8 connection(s)
I20260809 23:17:56.662355 30589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:17:56.664590 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 30589
I20260809 23:17:56.664832 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:43841
--local_ip_for_outbound_sockets=127.2.76.194
--tserver_master_addrs=127.2.76.254:45141
--webserver_port=41207
--webserver_interface=127.2.76.194
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:17:56.666177 30683 log.cc:824] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:17:56.668645 30683 tablet_bootstrap.cc:492] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:17:56.669004 30683 tablet_bootstrap.cc:492] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d: Bootstrap complete.
I20260809 23:17:56.670298 30683 ts_tablet_manager.cc:1397] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d: Time spent bootstrapping tablet: real 0.015s user 0.015s sys 0.000s
I20260809 23:17:56.671598 30683 raft_consensus.cc:348] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:17:56.671886 30683 raft_consensus.cc:729] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1508c984feb44458692d0d59c45257d, State: Initialized, Role: FOLLOWER
I20260809 23:17:56.672041 30683 consensus_queue.cc:260] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:17:56.672377 30683 ts_tablet_manager.cc:1428] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260809 23:17:56.707634 30852 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:17:56.663010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:41469)
0809 23:17:56.663171 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:56.663207 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:56.663577 (+ 370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:56.707512 (+ 43935us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
W20260809 23:17:56.707873 30850 heartbeater.cc:644] Failed to heartbeat to 127.2.76.254:45141 (0 consecutive failures): Network error: Failed to ping master at 127.2.76.254:45141: Client connection negotiation failed: client connection to 127.2.76.254:45141: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 23:17:56.708185 30852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:56.708055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:48139)
0809 23:17:56.708129 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:56.708170 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 23:17:56.708480 30852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:56.708401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:32905)
0809 23:17:56.708442 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:56.708463 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 23:17:56.708590 30850 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 23:17:56.739087 30853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:17:56.739252 30853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:17:56.739293 30853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:17:56.740430 30853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:17:56.740482 30853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:17:56.740521 30853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:17:56.741760 30853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:43841
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=41207
--tserver_master_addrs=127.2.76.254:45141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:17:56.741978 30853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:17:56.742175 30853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:17:56.746290 30868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:56.746305 30873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:56.746357 30871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:56.746433 30853 server_base.cc:1034] running on GCE node
I20260809 23:17:56.746677 30853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:17:56.746868 30853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:17:56.747993 30853 hybrid_clock.cc:648] HybridClock initialized: now 1786317476747982 us; error 25 us; skew 500 ppm
Aug 09 23:17:56 dist-test-slave-7vjv krb5kdc[28627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317476, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:17:56.750077 30853 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:17:56.750432 30853 webserver.cc:460] Webserver started at http://127.2.76.194:41207/ using document root <none> and password file <none>
I20260809 23:17:56.750595 30853 fs_manager.cc:359] Metadata directory not provided
I20260809 23:17:56.750628 30853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:17:56.751626 30853 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:17:56.752061 30882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:56.752216 30853 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:56.752295 30853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "968a3fdff4a94c588859f9e93bd115b7"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "ac4dafe8a1d877298a20bb3399f674ca"
server_key_iv: "0f3388828e110b2c28f773b99bb2df9c"
server_key_version: "encryptionkey@0"
I20260809 23:17:56.752555 30853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:17:56.772878 30853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:17:56.774351 30853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:17:56.774482 30853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:17:56.774668 30853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:17:56.774917 30853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:17:56.774961 30853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:56.774991 30853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:17:56.775012 30853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:56.780241 30853 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:43841
I20260809 23:17:56.780285 31069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:43841 every 8 connection(s)
I20260809 23:17:56.780548 30853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:17:56.781306 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:56.781045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:50271)
0809 23:17:56.781150 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:56.781199 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":63,"threads_started":1}
W20260809 23:17:56.781608 31070 heartbeater.cc:644] Failed to heartbeat to 127.2.76.254:45141 (0 consecutive failures): Network error: Failed to ping master at 127.2.76.254:45141: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
I20260809 23:17:56.781843 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:56.781762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:37597)
0809 23:17:56.781804 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:56.781828 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 23:17:56.782086 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:56.782013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:39137)
0809 23:17:56.782050 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:56.782069 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 23:17:56.782169 31070 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 23:17:56.790166 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 30853
I20260809 23:17:56.790530 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:39451
--local_ip_for_outbound_sockets=127.2.76.195
--tserver_master_addrs=127.2.76.254:45141
--webserver_port=44641
--webserver_interface=127.2.76.195
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:17:56.864459 31074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:17:56.864621 31074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:17:56.864673 31074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:17:56.865731 31074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:17:56.865797 31074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:17:56.865850 31074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:17:56.867192 31074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:39451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=44641
--tserver_master_addrs=127.2.76.254:45141
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:17:56.867439 31074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:17:56.867662 31074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:17:56.872044 31084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:17:56.872066 31085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:56.872223 31074 server_base.cc:1034] running on GCE node
W20260809 23:17:56.872267 31089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:17:56.872429 31074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:17:56.872696 31074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:17:56.873847 31074 hybrid_clock.cc:648] HybridClock initialized: now 1786317476873844 us; error 28 us; skew 500 ppm
Aug 09 23:17:56 dist-test-slave-7vjv krb5kdc[28627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317476, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:17:56.875972 31074 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:17:56.876437 31074 webserver.cc:460] Webserver started at http://127.2.76.195:44641/ using document root <none> and password file <none>
I20260809 23:17:56.876663 31074 fs_manager.cc:359] Metadata directory not provided
I20260809 23:17:56.876724 31074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:17:56.877871 31074 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:17:56.878396 31097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:56.878558 31074 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:17:56.878647 31074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "108c8e78f0c64c3b9295f7f9991232ba"
format_stamp: "Formatted at 2026-08-09 23:17:15 on dist-test-slave-7vjv"
server_key: "013e0fe73180527fe6504179c3ebe2af"
server_key_iv: "4e514ac7c3b6d484a680918153a0a5a2"
server_key_version: "encryptionkey@0"
I20260809 23:17:56.878906 31074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:17:56.893970 31074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:17:56.895959 31074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:17:56.896150 31074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:17:56.896368 31074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:17:56.896632 31074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:17:56.896682 31074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:56.896719 31074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:17:56.896744 31074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:17:56.902004 31074 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:39451
I20260809 23:17:56.902060 31293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:39451 every 8 connection(s)
I20260809 23:17:56.902313 31074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:17:56.903071 31298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:56.902769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:51489)
0809 23:17:56.902902 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:56.902955 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
W20260809 23:17:56.903405 31294 heartbeater.cc:644] Failed to heartbeat to 127.2.76.254:45141 (0 consecutive failures): Network error: Failed to ping master at 127.2.76.254:45141: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
I20260809 23:17:56.903631 31298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:56.903552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:57521)
0809 23:17:56.903593 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:56.903617 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 23:17:56.903879 31298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:56.903811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:42759)
0809 23:17:56.903849 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:56.903868 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 23:17:56.903983 31294 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 23:17:56.905401 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 31074
I20260809 23:17:57.709093 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:57.708879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:35935)
0809 23:17:57.709007 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:57.709039 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
I20260809 23:17:57.782598 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:57.782412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:56513)
0809 23:17:57.782527 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:57.782555 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":50,"threads_started":1}
I20260809 23:17:57.904460 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:57.904293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:52113)
0809 23:17:57.904391 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:57.904419 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":50,"threads_started":1}
I20260809 23:17:58.318921 31306 raft_consensus.cc:482] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:17:58.319036 31306 raft_consensus.cc:504] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:17:58.319442 31306 leader_election.cc:288] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451), 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841)
Aug 09 23:17:58 dist-test-slave-7vjv krb5kdc[28627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317476, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.194@KRBTEST.COM
Aug 09 23:17:58 dist-test-slave-7vjv krb5kdc[28627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317476, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.195@KRBTEST.COM
I20260809 23:17:58.331228 31312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:58.319799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:33249 (local address 127.2.76.194:43841)
0809 23:17:58.319976 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:58.319984 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:58.320016 (+ 32us) server_negotiation.cc:407] Connection header received
0809 23:17:58.320581 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:58.320584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:58.320641 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:58.320811 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:17:58.321833 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:58.323034 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:58.324438 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:58.325062 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:58.325184 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:58.327943 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:17:58.327946 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:17:58.327946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:17:58.327968 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:17:58.329660 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:58.330117 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:58.330120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:58.330120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:58.330217 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:58.330450 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:58.330451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:58.330451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:58.330673 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:17:58.330747 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:58.331024 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 23:17:58.331132 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260809 23:17:58.331218 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:58.319573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:39451 (local address 127.2.76.193:46137)
0809 23:17:58.319702 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:58.319713 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:58.319774 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:58.320921 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:58.320924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:58.321467 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:17:58.321716 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:58.321723 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:58.323136 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:58.323136 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:58.324358 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:58.324362 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:58.325166 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:58.325168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:58.325320 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:58.325941 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:17:58.325956 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:17:58.328229 (+ 2273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:17:58.329849 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:58.329850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:58.329851 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:58.330076 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:58.330334 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:58.330334 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:58.330334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:58.330401 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:58.330767 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:17:58.330768 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:17:58.330865 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 23:17:58.330962 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 23:17:58.331130 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":98,"thread_start_us":70,"threads_started":1}
I20260809 23:17:58.331297 31310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:58.319729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:46137 (local address 127.2.76.195:39451)
0809 23:17:58.319920 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 23:17:58.319933 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 23:17:58.319971 (+ 38us) server_negotiation.cc:407] Connection header received
0809 23:17:58.320581 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:17:58.320584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:17:58.320641 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:17:58.320811 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:17:58.321833 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:58.323034 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:58.324438 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:17:58.325062 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:17:58.325178 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:58.328320 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:17:58.328323 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:17:58.328324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:17:58.328334 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:17:58.329767 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:58.330156 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:58.330158 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:58.330159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:58.330258 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:17:58.330480 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:17:58.330481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:17:58.330482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:17:58.330680 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:17:58.330769 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 23:17:58.331065 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 23:17:58.331202 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":72,"threads_started":1}
I20260809 23:17:58.331326 31309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:58.319642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:43841 (local address 127.2.76.193:33249)
0809 23:17:58.319806 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:58.319816 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:17:58.319874 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:17:58.320921 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:17:58.320924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:17:58.321467 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:17:58.321716 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:58.321723 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:58.323133 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:58.323134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:58.324358 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:17:58.324362 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:17:58.325168 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:17:58.325168 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:17:58.325320 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:17:58.325935 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:17:58.325956 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:17:58.327849 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:17:58.329742 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:58.329750 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:58.329752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:58.330028 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:58.330276 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:17:58.330277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:17:58.330277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:17:58.330389 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:17:58.330741 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:17:58.330748 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:17:58.330867 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 23:17:58.330962 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 23:17:58.331215 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":214,"thread_start_us":82,"threads_started":1}
W20260809 23:17:58.331941 30670 leader_election.cc:341] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451): Not found: Tablet not found: 2e5afb46a6eb4187b7580afd3cfb8fa6
W20260809 23:17:58.332329 30674 leader_election.cc:341] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841): Not found: Tablet not found: 2e5afb46a6eb4187b7580afd3cfb8fa6
I20260809 23:17:58.332387 30674 leader_election.cc:302] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1508c984feb44458692d0d59c45257d; no voters: 108c8e78f0c64c3b9295f7f9991232ba, 968a3fdff4a94c588859f9e93bd115b7
I20260809 23:17:58.332521 31306 raft_consensus.cc:2726] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:17:58.709791 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:58.709654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:38455)
0809 23:17:58.709727 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:58.709759 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 23:17:58.783119 31314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:58.782938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:34913)
0809 23:17:58.783043 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:58.783081 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":55,"threads_started":1}
I20260809 23:17:58.905102 31315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:58.904931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:37289)
0809 23:17:58.905040 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:58.905070 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":53,"threads_started":1}
I20260809 23:17:59.710618 31320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:59.710403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:55575)
0809 23:17:59.710546 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:59.710579 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":62,"threads_started":1}
I20260809 23:17:59.783677 31322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:59.783488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:40747)
0809 23:17:59.783606 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:59.783640 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":55,"threads_started":1}
I20260809 23:17:59.905783 31324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:17:59.905593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:33565)
0809 23:17:59.905718 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 23:17:59.905751 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":58,"threads_started":1}
I20260809 23:18:00.001740 31325 raft_consensus.cc:482] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:18:00.001845 31325 raft_consensus.cc:504] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:18:00.002022 31325 leader_election.cc:288] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451), 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841)
W20260809 23:18:00.002446 30670 leader_election.cc:341] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451): Not found: Tablet not found: 2e5afb46a6eb4187b7580afd3cfb8fa6
W20260809 23:18:00.002522 30674 leader_election.cc:341] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841): Not found: Tablet not found: 2e5afb46a6eb4187b7580afd3cfb8fa6
I20260809 23:18:00.002611 30674 leader_election.cc:302] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1508c984feb44458692d0d59c45257d; no voters: 108c8e78f0c64c3b9295f7f9991232ba, 968a3fdff4a94c588859f9e93bd115b7
I20260809 23:18:00.002698 31325 raft_consensus.cc:2726] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:18:00.711221 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:00.711032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:44787)
0809 23:18:00.711147 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:00.711181 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":62,"threads_started":1}
I20260809 23:18:00.784261 31327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:00.784054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:40801)
0809 23:18:00.784196 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:00.784226 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
I20260809 23:18:00.906492 31329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:00.906189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:45605)
0809 23:18:00.906424 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:00.906457 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":72,"threads_started":1}
I20260809 23:18:01.711974 31330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:01.711760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:55847)
0809 23:18:01.711898 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:01.711930 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260809 23:18:01.784792 31332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:01.784612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:36863)
0809 23:18:01.784729 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:01.784759 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260809 23:18:01.907166 31333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:01.906990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:50507)
0809 23:18:01.907100 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:01.907132 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":52,"threads_started":1}
I20260809 23:18:02.686167 31335 raft_consensus.cc:482] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:18:02.686250 31335 raft_consensus.cc:504] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:18:02.686403 31335 leader_election.cc:288] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451), 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841)
W20260809 23:18:02.686853 30674 leader_election.cc:341] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841): Not found: Tablet not found: 2e5afb46a6eb4187b7580afd3cfb8fa6
W20260809 23:18:02.686939 30670 leader_election.cc:341] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451): Not found: Tablet not found: 2e5afb46a6eb4187b7580afd3cfb8fa6
I20260809 23:18:02.686980 30670 leader_election.cc:302] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1508c984feb44458692d0d59c45257d; no voters: 108c8e78f0c64c3b9295f7f9991232ba, 968a3fdff4a94c588859f9e93bd115b7
I20260809 23:18:02.687073 31335 raft_consensus.cc:2726] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:18:02.712544 31337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:02.712353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:46419)
0809 23:18:02.712466 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:02.712508 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":53,"threads_started":1}
I20260809 23:18:02.785303 31338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:02.785132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:48533)
0809 23:18:02.785237 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:02.785269 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":47,"threads_started":1}
I20260809 23:18:02.907769 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:02.907565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:55301)
0809 23:18:02.907705 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:02.907736 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260809 23:18:03.713160 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:03.712944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:36763)
0809 23:18:03.713079 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:03.713112 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":70,"threads_started":1}
I20260809 23:18:03.785900 31343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:03.785704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:55333)
0809 23:18:03.785834 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:03.785867 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260809 23:18:03.908406 31345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:03.908213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:45337)
0809 23:18:03.908347 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:03.908371 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
I20260809 23:18:04.713903 31347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:04.713583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:39359)
0809 23:18:04.713806 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:04.713846 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":78,"threads_started":1}
I20260809 23:18:04.786494 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:04.786312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:51353)
0809 23:18:04.786432 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:04.786462 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
I20260809 23:18:04.908916 31349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:04.908742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:38189)
0809 23:18:04.908848 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:04.908880 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
I20260809 23:18:05.714593 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:05.714385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:42229)
0809 23:18:05.714517 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:05.714550 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260809 23:18:05.787160 31352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:05.786967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:52865)
0809 23:18:05.787096 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:05.787128 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260809 23:18:05.909471 31353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:05.909303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:60123)
0809 23:18:05.909406 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:05.909436 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":56,"threads_started":1}
I20260809 23:18:06.715396 31356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:06.715141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:57403)
0809 23:18:06.715312 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:06.715352 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":90,"threads_started":1}
I20260809 23:18:06.787843 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:06.787659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:38693)
0809 23:18:06.787779 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:06.787812 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260809 23:18:06.910128 31359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:06.909937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:53121)
0809 23:18:06.910064 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:06.910096 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260809 23:18:07.211673 31361 raft_consensus.cc:482] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:18:07.211762 31361 raft_consensus.cc:504] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:18:07.211956 31361 leader_election.cc:288] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451), 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841)
W20260809 23:18:07.212334 30674 leader_election.cc:341] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841): Not found: Tablet not found: 2e5afb46a6eb4187b7580afd3cfb8fa6
W20260809 23:18:07.212414 30670 leader_election.cc:341] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451): Not found: Tablet not found: 2e5afb46a6eb4187b7580afd3cfb8fa6
I20260809 23:18:07.212486 30670 leader_election.cc:302] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1508c984feb44458692d0d59c45257d; no voters: 108c8e78f0c64c3b9295f7f9991232ba, 968a3fdff4a94c588859f9e93bd115b7
I20260809 23:18:07.212589 31361 raft_consensus.cc:2726] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:18:07.716281 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:07.715925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:57597)
0809 23:18:07.716205 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:07.716239 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":80,"threads_started":1}
I20260809 23:18:07.788406 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:07.788246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:36579)
0809 23:18:07.788343 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:07.788373 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":49,"threads_started":1}
I20260809 23:18:07.910746 31366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:07.910575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:45285)
0809 23:18:07.910682 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:07.910712 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260809 23:18:08.716987 31368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:08.716789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:53897)
0809 23:18:08.716911 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:08.716945 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
I20260809 23:18:08.788990 31370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:08.788800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:54763)
0809 23:18:08.788924 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:08.788957 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":54,"threads_started":1}
I20260809 23:18:08.911414 31371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:08.911246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:52795)
0809 23:18:08.911352 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:08.911382 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":55,"threads_started":1}
I20260809 23:18:09.001919 31373 raft_consensus.cc:482] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:18:09.002002 31373 raft_consensus.cc:504] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "108c8e78f0c64c3b9295f7f9991232ba" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 39451 } } peers { permanent_uuid: "968a3fdff4a94c588859f9e93bd115b7" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 43841 } } peers { permanent_uuid: "f1508c984feb44458692d0d59c45257d" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 40647 } }
I20260809 23:18:09.002156 31373 leader_election.cc:288] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451), 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841)
W20260809 23:18:09.002540 30670 leader_election.cc:341] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 108c8e78f0c64c3b9295f7f9991232ba (127.2.76.195:39451): Not found: Tablet not found: 2e5afb46a6eb4187b7580afd3cfb8fa6
W20260809 23:18:09.002597 30674 leader_election.cc:341] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 968a3fdff4a94c588859f9e93bd115b7 (127.2.76.194:43841): Not found: Tablet not found: 2e5afb46a6eb4187b7580afd3cfb8fa6
I20260809 23:18:09.002626 30674 leader_election.cc:302] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1508c984feb44458692d0d59c45257d; no voters: 108c8e78f0c64c3b9295f7f9991232ba, 968a3fdff4a94c588859f9e93bd115b7
I20260809 23:18:09.002748 31373 raft_consensus.cc:2726] T 2e5afb46a6eb4187b7580afd3cfb8fa6 P f1508c984feb44458692d0d59c45257d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:18:09.717772 31375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:09.717543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:51935)
0809 23:18:09.717695 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:09.717729 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260809 23:18:09.789626 31376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:09.789437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:53073)
0809 23:18:09.789555 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:09.789590 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260809 23:18:09.912078 31377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:09.911892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:38361)
0809 23:18:09.912014 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:09.912045 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":53,"threads_started":1}
I20260809 23:18:10.718466 31378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:10.718217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:40441)
0809 23:18:10.718384 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:10.718421 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260809 23:18:10.790442 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:10.790258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:45489)
0809 23:18:10.790377 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:10.790408 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260809 23:18:10.912755 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:10.912568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:53779)
0809 23:18:10.912694 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:10.912724 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260809 23:18:11.719189 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:11.719010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.193:39299)
0809 23:18:11.719118 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:11.719147 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":54,"threads_started":1}
I20260809 23:18:11.791077 31385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:11.790883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.194:42265)
0809 23:18:11.791008 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:11.791042 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260809 23:18:11.911562 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 30589
I20260809 23:18:11.917152 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:11.916679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:45141 (local address 127.2.76.195:39351)
0809 23:18:11.917075 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:11.917107 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.254:45141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":52,"threads_started":1}
I20260809 23:18:11.923998 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 30853
I20260809 23:18:11.936913 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 31074
2026-08-09T23:18:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58172 ms)
[----------] 30 tests from SecurityITest (253647 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:11 dist-test-slave-7vjv krb5kdc[31392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:11 dist-test-slave-7vjv krb5kdc[31392](info): set up 2 sockets
Aug 09 23:18:11 dist-test-slave-7vjv krb5kdc[31392](info): commencing operation
krb5kdc: starting...
W20260809 23:18:13.085945 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:13 dist-test-slave-7vjv krb5kdc[31392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:18:13.189999 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:37675
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:38207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:37675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:13.263064 31412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:13.263221 31412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:13.263239 31412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:13.264153 31412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:13.264204 31412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:13.264217 31412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:13.264230 31412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:13.264242 31412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:13.265615 31412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:38207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:37675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:37675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:13.265775 31412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:13.265915 31412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:18:13.270299 31412 server_base.cc:1034] running on GCE node
W20260809 23:18:13.270278 31418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:13.270284 31433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:13.270280 31424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:13.270691 31412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:13.270910 31412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:13.272068 31412 hybrid_clock.cc:648] HybridClock initialized: now 1786317493272059 us; error 25 us; skew 500 ppm
Aug 09 23:18:13 dist-test-slave-7vjv krb5kdc[31392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317493, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:13.277583 31412 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:18:13.278033 31412 webserver.cc:460] Webserver started at http://127.2.76.254:39297/ using document root <none> and password file <none>
I20260809 23:18:13.278254 31412 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:13.278319 31412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:13.278486 31412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:13.279486 31412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "4f2407334def40c3a4b5703e526fba5e"
format_stamp: "Formatted at 2026-08-09 23:18:13 on dist-test-slave-7vjv"
server_key: "490df717e9815fc7ce62e82bfca923f7"
server_key_iv: "b6a9717fae381148e3bbb5aa3f0385c4"
server_key_version: "encryptionkey@0"
I20260809 23:18:13.279850 31412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "4f2407334def40c3a4b5703e526fba5e"
format_stamp: "Formatted at 2026-08-09 23:18:13 on dist-test-slave-7vjv"
server_key: "490df717e9815fc7ce62e82bfca923f7"
server_key_iv: "b6a9717fae381148e3bbb5aa3f0385c4"
server_key_version: "encryptionkey@0"
I20260809 23:18:13.281172 31412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:18:13.281801 31443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:13.281955 31412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:13.282051 31412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "4f2407334def40c3a4b5703e526fba5e"
format_stamp: "Formatted at 2026-08-09 23:18:13 on dist-test-slave-7vjv"
server_key: "490df717e9815fc7ce62e82bfca923f7"
server_key_iv: "b6a9717fae381148e3bbb5aa3f0385c4"
server_key_version: "encryptionkey@0"
I20260809 23:18:13.282125 31412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:13.298720 31412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:13.300429 31412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:13.300570 31412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:13.304412 31412 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:37675
I20260809 23:18:13.304454 31531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:37675 every 8 connection(s)
I20260809 23:18:13.304797 31412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:13.305598 31532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:13.307521 31532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e: Bootstrap starting.
I20260809 23:18:13.308068 31532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:13.308365 31532 log.cc:824] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:13.309005 31532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e: No bootstrap required, opened a new log
I20260809 23:18:13.310374 31532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2407334def40c3a4b5703e526fba5e" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37675 } }
I20260809 23:18:13.310489 31532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:13.310554 31532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f2407334def40c3a4b5703e526fba5e, State: Initialized, Role: FOLLOWER
I20260809 23:18:13.310698 31532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2407334def40c3a4b5703e526fba5e" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37675 } }
I20260809 23:18:13.310770 31532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:13.310809 31532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:13.310853 31532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:13.311287 31532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2407334def40c3a4b5703e526fba5e" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37675 } }
I20260809 23:18:13.311386 31532 leader_election.cc:302] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f2407334def40c3a4b5703e526fba5e; no voters:
I20260809 23:18:13.311542 31532 leader_election.cc:288] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:13.311590 31535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:13.311757 31535 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [term 1 LEADER]: Becoming Leader. State: Replica: 4f2407334def40c3a4b5703e526fba5e, State: Running, Role: LEADER
I20260809 23:18:13.311822 31532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:13.311903 31535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2407334def40c3a4b5703e526fba5e" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37675 } }
I20260809 23:18:13.312348 31537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f2407334def40c3a4b5703e526fba5e. Latest consensus state: current_term: 1 leader_uuid: "4f2407334def40c3a4b5703e526fba5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2407334def40c3a4b5703e526fba5e" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37675 } } }
I20260809 23:18:13.312455 31536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f2407334def40c3a4b5703e526fba5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f2407334def40c3a4b5703e526fba5e" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37675 } } }
I20260809 23:18:13.312598 31536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:13.312489 31537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:13.312856 31550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:13.313311 31550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:13.314136 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 31412
I20260809 23:18:13.314234 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:13.314400 2355 external_mini_cluster.cc:1428] Setting key 6327dd3dc3ab75ede448c201d68309dd
I20260809 23:18:13.314741 31550 catalog_manager.cc:1347] Generated new cluster ID: 6e7e82ff99054734bb5b98d36fcd960b
I20260809 23:18:13.314786 31550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:18:13 dist-test-slave-7vjv krb5kdc[31392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:13.328029 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.314955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56680 (local address 127.2.76.254:37675)
0809 23:18:13.315100 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:13.315109 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:13.320381 (+ 5272us) server_negotiation.cc:407] Connection header received
0809 23:18:13.320489 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:13.320493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:13.320536 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:13.320652 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:13.321250 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.322046 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.322719 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.323093 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.323200 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.325256 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:13.325260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:13.325260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:13.325275 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:13.326701 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:13.327025 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:13.327028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:13.327028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:13.327104 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:13.327295 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:13.327296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:13.327296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:13.327496 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:13.327626 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:13.327724 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 23:18:13.327845 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260809 23:18:13.334718 31550 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:13.335235 31550 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:13.341606 31550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e: Generated new TSK 0
I20260809 23:18:13.341748 31550 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:18:13.375186 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37675
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:38207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:13.451527 31568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:13.451711 31568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:13.451735 31568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:13.452955 31568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:13.452997 31568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:13.453032 31568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:18:13.454362 31568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:38207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:13.454524 31568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:13.454681 31568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:13.459422 31574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:13.459452 31578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:13.459439 31575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:13.459815 31568 server_base.cc:1034] running on GCE node
I20260809 23:18:13.460049 31568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:13.460317 31568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:13.461455 31568 hybrid_clock.cc:648] HybridClock initialized: now 1786317493461442 us; error 30 us; skew 500 ppm
Aug 09 23:18:13 dist-test-slave-7vjv krb5kdc[31392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317493, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:13.463953 31568 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:18:13.464394 31568 webserver.cc:460] Webserver started at http://127.2.76.193:34113/ using document root <none> and password file <none>
I20260809 23:18:13.464579 31568 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:13.464617 31568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:13.464699 31568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:13.465601 31568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "18daa9dc6022405fa4c8040aba4a733a"
format_stamp: "Formatted at 2026-08-09 23:18:13 on dist-test-slave-7vjv"
server_key: "90a2bb6f36d20e42eda09c843b17f24e"
server_key_iv: "3b518673e85cd9207ff1175bfe7fb593"
server_key_version: "encryptionkey@0"
I20260809 23:18:13.465893 31568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "18daa9dc6022405fa4c8040aba4a733a"
format_stamp: "Formatted at 2026-08-09 23:18:13 on dist-test-slave-7vjv"
server_key: "90a2bb6f36d20e42eda09c843b17f24e"
server_key_iv: "3b518673e85cd9207ff1175bfe7fb593"
server_key_version: "encryptionkey@0"
I20260809 23:18:13.467229 31568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:18:13.467913 31596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:13.468132 31568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:18:13.468240 31568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "18daa9dc6022405fa4c8040aba4a733a"
format_stamp: "Formatted at 2026-08-09 23:18:13 on dist-test-slave-7vjv"
server_key: "90a2bb6f36d20e42eda09c843b17f24e"
server_key_iv: "3b518673e85cd9207ff1175bfe7fb593"
server_key_version: "encryptionkey@0"
I20260809 23:18:13.468323 31568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:13.480489 31568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:13.482651 31568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:13.482815 31568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:13.483017 31568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:13.483282 31568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:13.483332 31568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:13.483367 31568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:13.483393 31568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:13.489187 31568 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:46719
I20260809 23:18:13.489233 31777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:46719 every 8 connection(s)
I20260809 23:18:13.489504 31568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:18:13 dist-test-slave-7vjv krb5kdc[31392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317493, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:13.497013 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.490160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:45591 (local address 127.2.76.254:37675)
0809 23:18:13.490228 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:13.490230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:13.490365 (+ 135us) server_negotiation.cc:407] Connection header received
0809 23:18:13.490780 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:13.490780 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:13.490791 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:13.490838 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:13.491603 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.492051 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.492940 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.493110 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.493153 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.494817 (+ 1664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:13.494818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:13.494818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:13.494824 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:13.495816 (+ 992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:13.496186 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:13.496187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:13.496187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:13.496232 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:13.496426 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:13.496426 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:13.496426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:13.496517 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:13.496587 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:13.496850 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 23:18:13.496926 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 23:18:13.497138 31785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.490110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37675 (local address 127.2.76.193:45591)
0809 23:18:13.490262 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:13.490292 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:13.490706 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:13.490881 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:13.490884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:13.491367 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:13.491533 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.491537 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.492161 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.492162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.492872 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.492874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.493176 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.493177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.493262 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.493608 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:13.493616 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:13.494741 (+ 1125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:13.495883 (+ 1142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:13.495889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:13.495891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:13.496083 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:13.496288 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:13.496288 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:13.496289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:13.496376 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:13.496584 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:13.496592 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:13.496725 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 23:18:13.496820 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 23:18:13.496950 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260809 23:18:13.497565 31782 heartbeater.cc:347] Connected to a master server at 127.2.76.254:37675
I20260809 23:18:13.497651 31782 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:13.497805 31782 heartbeater.cc:511] Master 127.2.76.254:37675 requested a full tablet report, sending...
I20260809 23:18:13.498294 31471 ts_manager.cc:194] Registered new tserver with Master: 18daa9dc6022405fa4c8040aba4a733a (127.2.76.193:46719)
I20260809 23:18:13.498805 31471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:45591
I20260809 23:18:13.499698 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 31568
I20260809 23:18:13.499812 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:18:13.499984 2355 external_mini_cluster.cc:1428] Setting key ba8891451cf82468c78ab6ae113dd864
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:18:13.524282 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37675
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:38207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:13.598317 31792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:13.598515 31792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:13.598551 31792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:13.599514 31792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:13.599588 31792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:13.599647 31792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:18:13.600998 31792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:38207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:13.601251 31792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:13.601455 31792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:13.606017 31801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:13.606025 31802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:13.606022 31805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:13.606276 31792 server_base.cc:1034] running on GCE node
I20260809 23:18:13.606468 31792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:13.606717 31792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:13.607892 31792 hybrid_clock.cc:648] HybridClock initialized: now 1786317493607880 us; error 41 us; skew 500 ppm
Aug 09 23:18:13 dist-test-slave-7vjv krb5kdc[31392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317493, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:13.610132 31792 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:18:13.610488 31792 webserver.cc:460] Webserver started at http://127.2.76.194:32875/ using document root <none> and password file <none>
I20260809 23:18:13.610689 31792 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:13.610749 31792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:13.610868 31792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:13.611796 31792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "915d086e3400475596cf1ae609a54853"
format_stamp: "Formatted at 2026-08-09 23:18:13 on dist-test-slave-7vjv"
server_key: "1e4d6226e47c3e3e1a4931891f278455"
server_key_iv: "57ce03e263d0be0348d2655b2701985a"
server_key_version: "encryptionkey@0"
I20260809 23:18:13.612181 31792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "915d086e3400475596cf1ae609a54853"
format_stamp: "Formatted at 2026-08-09 23:18:13 on dist-test-slave-7vjv"
server_key: "1e4d6226e47c3e3e1a4931891f278455"
server_key_iv: "57ce03e263d0be0348d2655b2701985a"
server_key_version: "encryptionkey@0"
I20260809 23:18:13.613544 31792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:18:13.614238 31820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:13.614399 31792 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:13.614499 31792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "915d086e3400475596cf1ae609a54853"
format_stamp: "Formatted at 2026-08-09 23:18:13 on dist-test-slave-7vjv"
server_key: "1e4d6226e47c3e3e1a4931891f278455"
server_key_iv: "57ce03e263d0be0348d2655b2701985a"
server_key_version: "encryptionkey@0"
I20260809 23:18:13.614586 31792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:13.631795 31792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:13.633388 31792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:13.633528 31792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:13.633714 31792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:13.633972 31792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:13.634029 31792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:13.634066 31792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:13.634092 31792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:13.639223 31792 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:46259
I20260809 23:18:13.639290 32020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:46259 every 8 connection(s)
I20260809 23:18:13.639564 31792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:18:13 dist-test-slave-7vjv krb5kdc[31392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317493, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:13.647002 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.640250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:57975 (local address 127.2.76.254:37675)
0809 23:18:13.640315 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:13.640317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:13.640475 (+ 158us) server_negotiation.cc:407] Connection header received
0809 23:18:13.640839 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:13.640840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:13.640848 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:13.640895 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:13.641621 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.642034 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.642915 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.643081 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.643126 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.644953 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:13.644954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:13.644955 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:13.644960 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:13.645962 (+ 1002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:13.646271 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:13.646272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:13.646272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:13.646316 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:13.646497 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:13.646498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:13.646498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:13.646587 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:13.646629 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:13.646872 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 23:18:13.646935 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 23:18:13.647130 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.640228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37675 (local address 127.2.76.194:57975)
0809 23:18:13.640360 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:13.640393 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:13.640766 (+ 373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:13.640935 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:13.640937 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:13.641387 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:13.641545 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.641548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.642105 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.642105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.642843 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.642844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.643148 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.643149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.643232 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.643680 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:13.643689 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:13.644878 (+ 1189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:13.646025 (+ 1147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:13.646031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:13.646033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:13.646211 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:13.646368 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:13.646368 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:13.646369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:13.646451 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:13.646634 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:13.646639 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:13.646724 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 23:18:13.646815 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 23:18:13.646938 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
I20260809 23:18:13.647617 32021 heartbeater.cc:347] Connected to a master server at 127.2.76.254:37675
I20260809 23:18:13.647714 32021 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:13.647897 32021 heartbeater.cc:511] Master 127.2.76.254:37675 requested a full tablet report, sending...
I20260809 23:18:13.648285 31471 ts_manager.cc:194] Registered new tserver with Master: 915d086e3400475596cf1ae609a54853 (127.2.76.194:46259)
I20260809 23:18:13.648648 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 31792
I20260809 23:18:13.648655 31471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:57975
I20260809 23:18:13.648741 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:18:13.648926 2355 external_mini_cluster.cc:1428] Setting key 3467480cce56141430631ba3350dae7f
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:18:13.672128 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37675
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:38207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:13.742934 32029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:13.743112 32029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:13.743148 32029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:13.744134 32029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:13.744191 32029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:13.744230 32029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:18:13.745524 32029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:38207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:13.745775 32029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:13.745980 32029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:18:13.750597 32029 server_base.cc:1034] running on GCE node
W20260809 23:18:13.750559 32037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:13.750591 32038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:13.750567 32042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:13.750986 32029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:13.751178 32029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:13.752302 32029 hybrid_clock.cc:648] HybridClock initialized: now 1786317493752294 us; error 30 us; skew 500 ppm
Aug 09 23:18:13 dist-test-slave-7vjv krb5kdc[31392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317493, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:13.754391 32029 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:18:13.754750 32029 webserver.cc:460] Webserver started at http://127.2.76.195:36817/ using document root <none> and password file <none>
I20260809 23:18:13.754949 32029 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:13.754987 32029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:13.755108 32029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:13.755952 32029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "9a8d76cdfdf3488cadf3795d5d62903e"
format_stamp: "Formatted at 2026-08-09 23:18:13 on dist-test-slave-7vjv"
server_key: "b326d466af9c211ebcdd860006743277"
server_key_iv: "ba1213a9ff43403d79c4180e147b2942"
server_key_version: "encryptionkey@0"
I20260809 23:18:13.756271 32029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "9a8d76cdfdf3488cadf3795d5d62903e"
format_stamp: "Formatted at 2026-08-09 23:18:13 on dist-test-slave-7vjv"
server_key: "b326d466af9c211ebcdd860006743277"
server_key_iv: "ba1213a9ff43403d79c4180e147b2942"
server_key_version: "encryptionkey@0"
I20260809 23:18:13.757364 32029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:18:13.757956 32055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:13.758150 32029 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:18:13.758229 32029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "9a8d76cdfdf3488cadf3795d5d62903e"
format_stamp: "Formatted at 2026-08-09 23:18:13 on dist-test-slave-7vjv"
server_key: "b326d466af9c211ebcdd860006743277"
server_key_iv: "ba1213a9ff43403d79c4180e147b2942"
server_key_version: "encryptionkey@0"
I20260809 23:18:13.758289 32029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:13.771042 32029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:13.773216 32029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:13.773307 32029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:13.773433 32029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:13.773677 32029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:13.773705 32029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:13.773722 32029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:13.773734 32029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:13.779196 32029 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:33617
I20260809 23:18:13.779258 32237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:33617 every 8 connection(s)
I20260809 23:18:13.779525 32029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:18:13 dist-test-slave-7vjv krb5kdc[31392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317493, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:13.786087 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 32029
I20260809 23:18:13.786201 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:18:13.786386 2355 external_mini_cluster.cc:1428] Setting key 990cfe4c85b60b3496f7ac2a2c5e185d
I20260809 23:18:13.787941 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.780249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:35217 (local address 127.2.76.254:37675)
0809 23:18:13.780307 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:13.780309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:13.780483 (+ 174us) server_negotiation.cc:407] Connection header received
0809 23:18:13.780839 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:13.780840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:13.780849 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:13.780905 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:13.781694 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.782134 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.783054 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.783225 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.783274 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.785148 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:13.785150 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:13.785150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:13.785156 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:13.786639 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:13.787116 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:13.787117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:13.787117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:13.787181 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:13.787423 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:13.787424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:13.787424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:13.787542 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:13.787593 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:13.787813 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 23:18:13.787868 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 23:18:13.788096 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.780232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37675 (local address 127.2.76.195:35217)
0809 23:18:13.780395 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:13.780423 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:13.780762 (+ 339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:13.780944 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:13.780946 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:13.781381 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:13.781536 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.781540 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.782213 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.782213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.782978 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.782980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.783296 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.783297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.783393 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.783799 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:13.783806 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:13.785069 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:13.786763 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:13.786771 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:13.786772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:13.787055 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:13.787244 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:13.787245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:13.787245 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:13.787364 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:13.787607 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:13.787611 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:13.787702 (+ 91us) client_negotiation.cc:769] Sending connection context
0809 23:18:13.787774 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 23:18:13.787881 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":110,"threads_started":1}
I20260809 23:18:13.788540 32238 heartbeater.cc:347] Connected to a master server at 127.2.76.254:37675
I20260809 23:18:13.788631 32238 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:13.788779 32238 heartbeater.cc:511] Master 127.2.76.254:37675 requested a full tablet report, sending...
I20260809 23:18:13.789108 31471 ts_manager.cc:194] Registered new tserver with Master: 9a8d76cdfdf3488cadf3795d5d62903e (127.2.76.195:33617)
I20260809 23:18:13.789405 31471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:35217
I20260809 23:18:13.797856 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:18:13.804045 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.799066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56690 (local address 127.2.76.254:37675)
0809 23:18:13.799114 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:13.799116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:13.799203 (+ 87us) server_negotiation.cc:407] Connection header received
0809 23:18:13.799234 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:13.799235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:13.799241 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:13.799277 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:13.799767 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.800220 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.800882 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.801030 (+ 148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.801072 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.802044 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:13.802046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:13.802046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:13.802052 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:13.802965 (+ 913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:13.803302 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:13.803303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:13.803303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:13.803356 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:13.803547 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:13.803548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:13.803548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:13.803667 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:13.803763 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:13.803876 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 23:18:13.803948 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 23:18:13.805613 31471 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:18:13.807147 31471 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:18:13.812844 32252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.809113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:46719 (local address 127.0.0.1:34780)
0809 23:18:13.809363 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:13.809374 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:13.809434 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:13.809757 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:13.809760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:13.809762 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:13.809825 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.809826 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.810878 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.810879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.811787 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.811790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.812581 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.812582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.812655 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.812667 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 23:18:13.812728 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 23:18:13.812775 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":66,"threads_started":1}
I20260809 23:18:13.812970 32257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.809398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34780 (local address 127.2.76.193:46719)
0809 23:18:13.809603 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:13.809607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:13.809614 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:18:13.809649 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:13.809651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:13.809662 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:13.809752 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:13.809913 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.810781 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.811876 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.812501 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.812579 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.812630 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:13.812752 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 23:18:13.812886 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":92,"threads_started":1}
I20260809 23:18:13.813279 32253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.809143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:33617 (local address 127.0.0.1:40768)
0809 23:18:13.809848 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:13.809858 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:13.809908 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:13.810165 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:13.810166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:13.810168 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:13.810212 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.810214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.811321 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.811322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.812130 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.812133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.813059 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.813060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.813114 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.813118 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:18:13.813156 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 23:18:13.813186 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":2,"thread_start_us":60,"threads_started":1}
I20260809 23:18:13.813279 32254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.809090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:46259 (local address 127.0.0.1:41960)
0809 23:18:13.809277 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:13.809358 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:13.809429 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:13.809936 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:13.809937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:13.809940 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:13.809987 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.809989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.811298 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.811299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.812264 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.812267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.813082 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.813082 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.813123 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.813126 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:18:13.813151 (+ 25us) client_negotiation.cc:240] Negotiation successful
0809 23:18:13.813186 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260809 23:18:13.813495 32255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.809377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40768 (local address 127.2.76.195:33617)
0809 23:18:13.809773 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:13.809778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:13.809964 (+ 186us) server_negotiation.cc:407] Connection header received
0809 23:18:13.810014 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:13.810016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:13.810031 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:13.810098 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:13.810293 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.811231 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.812192 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.812795 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.812940 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.812998 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:13.813386 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 23:18:13.813430 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":74,"threads_started":1}
I20260809 23:18:13.813804 32258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.809551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41960 (local address 127.2.76.194:46259)
0809 23:18:13.809730 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:13.809735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:13.809746 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:18:13.809788 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:13.809790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:13.809805 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:13.810112 (+ 307us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:13.810195 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.811194 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.812344 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.812943 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.813421 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.813479 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:13.813503 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 23:18:13.813545 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260809 23:18:13.813843 31676 tablet_service.cc:1467] Processing CreateTablet for tablet d6d5d8f9c9464be4a4c4533e1b7f2b5e (DEFAULT_TABLE table=test-table [id=f59b716c90a4466badd04a9ed5011cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:13.814205 31676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6d5d8f9c9464be4a4c4533e1b7f2b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:13.814109 32131 tablet_service.cc:1467] Processing CreateTablet for tablet d6d5d8f9c9464be4a4c4533e1b7f2b5e (DEFAULT_TABLE table=test-table [id=f59b716c90a4466badd04a9ed5011cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:13.814409 32131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6d5d8f9c9464be4a4c4533e1b7f2b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:13.814344 31905 tablet_service.cc:1467] Processing CreateTablet for tablet d6d5d8f9c9464be4a4c4533e1b7f2b5e (DEFAULT_TABLE table=test-table [id=f59b716c90a4466badd04a9ed5011cf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:13.814604 31905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6d5d8f9c9464be4a4c4533e1b7f2b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:13.816536 32259 tablet_bootstrap.cc:492] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e: Bootstrap starting.
I20260809 23:18:13.817303 32260 tablet_bootstrap.cc:492] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 18daa9dc6022405fa4c8040aba4a733a: Bootstrap starting.
I20260809 23:18:13.817386 32259 tablet_bootstrap.cc:654] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:13.817704 32261 tablet_bootstrap.cc:492] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853: Bootstrap starting.
I20260809 23:18:13.817984 32259 log.cc:824] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:13.818071 32260 tablet_bootstrap.cc:654] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 18daa9dc6022405fa4c8040aba4a733a: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:13.818368 32261 tablet_bootstrap.cc:654] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:13.818629 32260 log.cc:824] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 18daa9dc6022405fa4c8040aba4a733a: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:13.818809 32259 tablet_bootstrap.cc:492] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e: No bootstrap required, opened a new log
I20260809 23:18:13.818903 32259 ts_tablet_manager.cc:1397] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:18:13.818945 32261 log.cc:824] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:13.819329 32260 tablet_bootstrap.cc:492] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 18daa9dc6022405fa4c8040aba4a733a: No bootstrap required, opened a new log
I20260809 23:18:13.819393 32260 ts_tablet_manager.cc:1397] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 18daa9dc6022405fa4c8040aba4a733a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:18:13.819646 32261 tablet_bootstrap.cc:492] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853: No bootstrap required, opened a new log
I20260809 23:18:13.819720 32261 ts_tablet_manager.cc:1397] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:18:13.820708 32259 raft_consensus.cc:348] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d086e3400475596cf1ae609a54853" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 46259 } } peers { permanent_uuid: "18daa9dc6022405fa4c8040aba4a733a" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46719 } } peers { permanent_uuid: "9a8d76cdfdf3488cadf3795d5d62903e" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33617 } }
I20260809 23:18:13.820890 32259 raft_consensus.cc:374] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:13.820931 32259 raft_consensus.cc:729] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8d76cdfdf3488cadf3795d5d62903e, State: Initialized, Role: FOLLOWER
I20260809 23:18:13.821084 32259 consensus_queue.cc:260] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d086e3400475596cf1ae609a54853" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 46259 } } peers { permanent_uuid: "18daa9dc6022405fa4c8040aba4a733a" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46719 } } peers { permanent_uuid: "9a8d76cdfdf3488cadf3795d5d62903e" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33617 } }
I20260809 23:18:13.821352 32259 ts_tablet_manager.cc:1428] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:18:13.821318 32260 raft_consensus.cc:348] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 18daa9dc6022405fa4c8040aba4a733a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d086e3400475596cf1ae609a54853" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 46259 } } peers { permanent_uuid: "18daa9dc6022405fa4c8040aba4a733a" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46719 } } peers { permanent_uuid: "9a8d76cdfdf3488cadf3795d5d62903e" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33617 } }
I20260809 23:18:13.821466 32238 heartbeater.cc:503] Master 127.2.76.254:37675 was elected leader, sending a full tablet report...
I20260809 23:18:13.821480 32260 raft_consensus.cc:374] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 18daa9dc6022405fa4c8040aba4a733a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:13.821532 32260 raft_consensus.cc:729] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 18daa9dc6022405fa4c8040aba4a733a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18daa9dc6022405fa4c8040aba4a733a, State: Initialized, Role: FOLLOWER
I20260809 23:18:13.821444 32261 raft_consensus.cc:348] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d086e3400475596cf1ae609a54853" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 46259 } } peers { permanent_uuid: "18daa9dc6022405fa4c8040aba4a733a" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46719 } } peers { permanent_uuid: "9a8d76cdfdf3488cadf3795d5d62903e" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33617 } }
I20260809 23:18:13.821600 32261 raft_consensus.cc:374] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:13.821655 32261 raft_consensus.cc:729] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915d086e3400475596cf1ae609a54853, State: Initialized, Role: FOLLOWER
I20260809 23:18:13.821637 32260 consensus_queue.cc:260] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 18daa9dc6022405fa4c8040aba4a733a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d086e3400475596cf1ae609a54853" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 46259 } } peers { permanent_uuid: "18daa9dc6022405fa4c8040aba4a733a" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46719 } } peers { permanent_uuid: "9a8d76cdfdf3488cadf3795d5d62903e" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33617 } }
I20260809 23:18:13.821774 32261 consensus_queue.cc:260] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d086e3400475596cf1ae609a54853" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 46259 } } peers { permanent_uuid: "18daa9dc6022405fa4c8040aba4a733a" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46719 } } peers { permanent_uuid: "9a8d76cdfdf3488cadf3795d5d62903e" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33617 } }
I20260809 23:18:13.821898 32260 ts_tablet_manager.cc:1428] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 18daa9dc6022405fa4c8040aba4a733a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:18:13.822116 32261 ts_tablet_manager.cc:1428] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:18:13.822124 31782 heartbeater.cc:503] Master 127.2.76.254:37675 was elected leader, sending a full tablet report...
I20260809 23:18:13.822145 32021 heartbeater.cc:503] Master 127.2.76.254:37675 was elected leader, sending a full tablet report...
W20260809 23:18:13.890179 32023 tablet.cc:2369] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:18:13.943526 32275 raft_consensus.cc:482] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:18:13.943642 32275 raft_consensus.cc:504] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d086e3400475596cf1ae609a54853" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 46259 } } peers { permanent_uuid: "18daa9dc6022405fa4c8040aba4a733a" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46719 } } peers { permanent_uuid: "9a8d76cdfdf3488cadf3795d5d62903e" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33617 } }
I20260809 23:18:13.943956 32275 leader_election.cc:288] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18daa9dc6022405fa4c8040aba4a733a (127.2.76.193:46719), 9a8d76cdfdf3488cadf3795d5d62903e (127.2.76.195:33617)
I20260809 23:18:13.946619 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.944132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:33617 (local address 127.2.76.194:44333)
0809 23:18:13.944183 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:13.944194 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:13.944234 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:13.944391 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:13.944392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:13.944394 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:13.944475 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.944477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.945078 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.945079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.945789 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.945792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.946398 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.946399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.946467 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.946470 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:18:13.946522 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 23:18:13.946555 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":28}
I20260809 23:18:13.946769 32257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.944194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:58985 (local address 127.2.76.193:46719)
0809 23:18:13.944273 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:13.944275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:13.944396 (+ 121us) server_negotiation.cc:407] Connection header received
0809 23:18:13.944442 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:13.944443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:13.944457 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:13.944522 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:13.944665 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.945088 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.945932 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.946381 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.946444 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.946451 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:13.946660 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 23:18:13.946696 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 23:18:13.946954 32279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.944159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:46719 (local address 127.2.76.194:58985)
0809 23:18:13.944335 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:13.944343 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:13.944385 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:13.944536 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:13.944537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:13.944538 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:13.944600 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.944601 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.945161 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.945162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.945811 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:13.945813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.946485 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.946486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:13.946587 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.946590 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:18:13.946848 (+ 258us) client_negotiation.cc:240] Negotiation successful
0809 23:18:13.946887 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":110,"threads_started":1}
I20260809 23:18:13.947069 32255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.944202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:44333 (local address 127.2.76.195:33617)
0809 23:18:13.944258 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:13.944260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:13.944268 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:18:13.944294 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:13.944295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:13.944305 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:13.944363 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:13.944543 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.945011 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.945890 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.946295 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.946358 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.946367 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:13.946940 (+ 573us) server_negotiation.cc:299] Negotiation successful
0809 23:18:13.946983 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 23:18:13.947299 31714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6d5d8f9c9464be4a4c4533e1b7f2b5e" candidate_uuid: "915d086e3400475596cf1ae609a54853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18daa9dc6022405fa4c8040aba4a733a" is_pre_election: true
I20260809 23:18:13.947316 32171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6d5d8f9c9464be4a4c4533e1b7f2b5e" candidate_uuid: "915d086e3400475596cf1ae609a54853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8d76cdfdf3488cadf3795d5d62903e" is_pre_election: true
I20260809 23:18:13.947440 31714 raft_consensus.cc:2454] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 18daa9dc6022405fa4c8040aba4a733a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915d086e3400475596cf1ae609a54853 in term 0.
I20260809 23:18:13.947440 32171 raft_consensus.cc:2454] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 915d086e3400475596cf1ae609a54853 in term 0.
I20260809 23:18:13.947727 31825 leader_election.cc:302] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 915d086e3400475596cf1ae609a54853, 9a8d76cdfdf3488cadf3795d5d62903e; no voters:
I20260809 23:18:13.947880 32275 raft_consensus.cc:2781] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:18:13.947939 32275 raft_consensus.cc:482] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:18:13.947963 32275 raft_consensus.cc:3037] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:13.948681 32275 raft_consensus.cc:504] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d086e3400475596cf1ae609a54853" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 46259 } } peers { permanent_uuid: "18daa9dc6022405fa4c8040aba4a733a" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46719 } } peers { permanent_uuid: "9a8d76cdfdf3488cadf3795d5d62903e" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33617 } }
I20260809 23:18:13.948824 32275 leader_election.cc:288] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [CANDIDATE]: Term 1 election: Requested vote from peers 18daa9dc6022405fa4c8040aba4a733a (127.2.76.193:46719), 9a8d76cdfdf3488cadf3795d5d62903e (127.2.76.195:33617)
I20260809 23:18:13.948962 31714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6d5d8f9c9464be4a4c4533e1b7f2b5e" candidate_uuid: "915d086e3400475596cf1ae609a54853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18daa9dc6022405fa4c8040aba4a733a"
I20260809 23:18:13.948985 32171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6d5d8f9c9464be4a4c4533e1b7f2b5e" candidate_uuid: "915d086e3400475596cf1ae609a54853" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8d76cdfdf3488cadf3795d5d62903e"
I20260809 23:18:13.949052 32171 raft_consensus.cc:3037] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:13.949057 31714 raft_consensus.cc:3037] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 18daa9dc6022405fa4c8040aba4a733a [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:13.949760 32171 raft_consensus.cc:2454] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915d086e3400475596cf1ae609a54853 in term 1.
I20260809 23:18:13.949826 31714 raft_consensus.cc:2454] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 18daa9dc6022405fa4c8040aba4a733a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 915d086e3400475596cf1ae609a54853 in term 1.
I20260809 23:18:13.949980 31832 leader_election.cc:302] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18daa9dc6022405fa4c8040aba4a733a, 915d086e3400475596cf1ae609a54853; no voters:
I20260809 23:18:13.950133 32275 raft_consensus.cc:2781] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:13.950313 32275 raft_consensus.cc:686] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [term 1 LEADER]: Becoming Leader. State: Replica: 915d086e3400475596cf1ae609a54853, State: Running, Role: LEADER
I20260809 23:18:13.950511 32275 consensus_queue.cc:237] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d086e3400475596cf1ae609a54853" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 46259 } } peers { permanent_uuid: "18daa9dc6022405fa4c8040aba4a733a" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46719 } } peers { permanent_uuid: "9a8d76cdfdf3488cadf3795d5d62903e" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33617 } }
I20260809 23:18:13.951252 31471 catalog_manager.cc:5495] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 reported cstate change: term changed from 0 to 1, leader changed from <none> to 915d086e3400475596cf1ae609a54853 (127.2.76.194). New cstate: current_term: 1 leader_uuid: "915d086e3400475596cf1ae609a54853" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915d086e3400475596cf1ae609a54853" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 46259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "18daa9dc6022405fa4c8040aba4a733a" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8d76cdfdf3488cadf3795d5d62903e" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33617 } health_report { overall_health: UNKNOWN } } }
W20260809 23:18:13.990168 31783 tablet.cc:2369] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 18daa9dc6022405fa4c8040aba4a733a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:18:13.991403 32258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:13.989365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41962 (local address 127.2.76.194:46259)
0809 23:18:13.989416 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:13.989418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:13.989425 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:18:13.989487 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:13.989488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:13.989514 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:13.989585 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:13.989705 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.990131 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.990775 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:13.990952 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:13.991001 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:13.991128 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:13.991210 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:13.991262 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:13.991290 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 23:18:13.991304 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 23:18:13.993038 32171 raft_consensus.cc:1261] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e [term 1 FOLLOWER]: Refusing update from remote peer 915d086e3400475596cf1ae609a54853: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:13.993037 31714 raft_consensus.cc:1261] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 18daa9dc6022405fa4c8040aba4a733a [term 1 FOLLOWER]: Refusing update from remote peer 915d086e3400475596cf1ae609a54853: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:13.993360 32282 consensus_queue.cc:1035] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8d76cdfdf3488cadf3795d5d62903e" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:13.993465 32275 consensus_queue.cc:1035] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [LEADER]: Connected to new peer: Peer: permanent_uuid: "18daa9dc6022405fa4c8040aba4a733a" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:13.998093 31469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:18:13.998220 31469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:18:13.998917 31469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f2407334def40c3a4b5703e526fba5e: Sending DeleteTablet for 3 replicas of tablet d6d5d8f9c9464be4a4c4533e1b7f2b5e
I20260809 23:18:13.999255 31905 tablet_service.cc:1514] Processing DeleteTablet for tablet d6d5d8f9c9464be4a4c4533e1b7f2b5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:13 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:41960
I20260809 23:18:13.999289 32131 tablet_service.cc:1514] Processing DeleteTablet for tablet d6d5d8f9c9464be4a4c4533e1b7f2b5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:13 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:40768
I20260809 23:18:13.999310 31676 tablet_service.cc:1514] Processing DeleteTablet for tablet d6d5d8f9c9464be4a4c4533e1b7f2b5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:13 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:34780
I20260809 23:18:13.999491 32316 tablet_replica.cc:330] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e: stopping tablet replica
I20260809 23:18:13.999608 32316 raft_consensus.cc:2229] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:18:13.999665 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 31568
I20260809 23:18:14.017525 32316 raft_consensus.cc:2258] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:18:14.019418 32316 ts_tablet_manager.cc:1905] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:18:14.021072 32316 ts_tablet_manager.cc:1918] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:18:14.021142 32316 log.cc:1196] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/d6d5d8f9c9464be4a4c4533e1b7f2b5e
I20260809 23:18:14.021404 32316 ts_tablet_manager.cc:1939] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 9a8d76cdfdf3488cadf3795d5d62903e: Deleting consensus metadata
I20260809 23:18:14.021855 31447 catalog_manager.cc:4841] TS 9a8d76cdfdf3488cadf3795d5d62903e (127.2.76.195:33617): tablet d6d5d8f9c9464be4a4c4533e1b7f2b5e (table test-table [id=f59b716c90a4466badd04a9ed5011cf7]) successfully deleted
I20260809 23:18:14.027064 32310 tablet_replica.cc:330] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853: stopping tablet replica
I20260809 23:18:14.027174 32310 raft_consensus.cc:2229] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:18:14.027269 32310 raft_consensus.cc:2258] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:18:14.027645 32310 ts_tablet_manager.cc:1905] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:18:14.029043 32310 ts_tablet_manager.cc:1918] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:18:14.029127 32310 log.cc:1196] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/d6d5d8f9c9464be4a4c4533e1b7f2b5e
I20260809 23:18:14.029379 32310 ts_tablet_manager.cc:1939] T d6d5d8f9c9464be4a4c4533e1b7f2b5e P 915d086e3400475596cf1ae609a54853: Deleting consensus metadata
I20260809 23:18:14.029731 31444 catalog_manager.cc:4841] TS 915d086e3400475596cf1ae609a54853 (127.2.76.194:46259): tablet d6d5d8f9c9464be4a4c4533e1b7f2b5e (table test-table [id=f59b716c90a4466badd04a9ed5011cf7]) successfully deleted
W20260809 23:18:14.032387 31450 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:46719 (error 108)
I20260809 23:18:14.032588 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 31792
I20260809 23:18:14.032835 32254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.032724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:46719 (local address 127.0.0.1:34792)
0809 23:18:14.032764 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:14.032798 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:46719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 23:18:14.032931 31450 catalog_manager.cc:4584] TS 18daa9dc6022405fa4c8040aba4a733a (127.2.76.193:46719): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d6d5d8f9c9464be4a4c4533e1b7f2b5e: Network error: Client connection negotiation failed: client connection to 127.2.76.193:46719: connect: Connection refused (error 111)
I20260809 23:18:14.041810 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 32029
I20260809 23:18:14.049324 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 31412
2026-08-09T23:18:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2108 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T23:18:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:14Z Disabled control of system clock
I20260809 23:18:14.079363 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:35165
--webserver_interface=127.2.76.254
--webserver_port=0
--builtin_ntp_servers=127.2.76.212:38483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:35165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:18:14.158318 32330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:14.158516 32330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:14.158538 32330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:14.159565 32330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:14.159605 32330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:14.159619 32330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:14.159632 32330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:14.159646 32330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:14.161024 32330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:38483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:35165
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:35165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:14.161208 32330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:14.161430 32330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:14.165781 32338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:14.165879 32335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:14.165982 32330 server_base.cc:1034] running on GCE node
W20260809 23:18:14.165782 32343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:14.166343 32330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:14.166622 32330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:14.167759 32330 hybrid_clock.cc:648] HybridClock initialized: now 1786317494167740 us; error 34 us; skew 500 ppm
I20260809 23:18:14.169090 32330 webserver.cc:460] Webserver started at http://127.2.76.254:45201/ using document root <none> and password file <none>
I20260809 23:18:14.169312 32330 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:14.169360 32330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:14.169528 32330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:14.170506 32330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "c516d74195c742429b3ebd5b33d4f3fa"
format_stamp: "Formatted at 2026-08-09 23:18:14 on dist-test-slave-7vjv"
server_key: "8351966a388c426c0f04eb25f7acd981"
server_key_iv: "e86db4837e6d2281764a391c0847ed70"
server_key_version: "encryptionkey@0"
I20260809 23:18:14.170847 32330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "c516d74195c742429b3ebd5b33d4f3fa"
format_stamp: "Formatted at 2026-08-09 23:18:14 on dist-test-slave-7vjv"
server_key: "8351966a388c426c0f04eb25f7acd981"
server_key_iv: "e86db4837e6d2281764a391c0847ed70"
server_key_version: "encryptionkey@0"
I20260809 23:18:14.172156 32330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:18:14.172780 32354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:14.172948 32330 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:18:14.173039 32330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "c516d74195c742429b3ebd5b33d4f3fa"
format_stamp: "Formatted at 2026-08-09 23:18:14 on dist-test-slave-7vjv"
server_key: "8351966a388c426c0f04eb25f7acd981"
server_key_iv: "e86db4837e6d2281764a391c0847ed70"
server_key_version: "encryptionkey@0"
I20260809 23:18:14.173112 32330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:14.183384 32330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:14.183598 32330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:14.183691 32330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:14.187809 32330 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:35165
I20260809 23:18:14.187839 32441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:35165 every 8 connection(s)
I20260809 23:18:14.188217 32330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:14.189102 32444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:14.190997 32444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa: Bootstrap starting.
I20260809 23:18:14.191480 32444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:14.191725 32444 log.cc:824] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:14.192368 32444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa: No bootstrap required, opened a new log
I20260809 23:18:14.193468 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 32330
I20260809 23:18:14.193601 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:14.193784 2355 external_mini_cluster.cc:1428] Setting key a97bbc4012a66846252ec10fdd86f3ab
I20260809 23:18:14.193922 32444 raft_consensus.cc:348] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c516d74195c742429b3ebd5b33d4f3fa" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35165 } }
I20260809 23:18:14.194093 32444 raft_consensus.cc:374] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:14.194432 32444 raft_consensus.cc:729] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c516d74195c742429b3ebd5b33d4f3fa, State: Initialized, Role: FOLLOWER
I20260809 23:18:14.194589 32444 consensus_queue.cc:260] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c516d74195c742429b3ebd5b33d4f3fa" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35165 } }
I20260809 23:18:14.194674 32444 raft_consensus.cc:388] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:14.194705 32444 raft_consensus.cc:482] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:14.194765 32444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:14.195405 32444 raft_consensus.cc:504] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c516d74195c742429b3ebd5b33d4f3fa" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35165 } }
I20260809 23:18:14.195519 32444 leader_election.cc:302] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c516d74195c742429b3ebd5b33d4f3fa; no voters:
I20260809 23:18:14.195664 32444 leader_election.cc:288] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:14.195729 32449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:14.195879 32444 sys_catalog.cc:564] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:14.195889 32449 raft_consensus.cc:686] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [term 1 LEADER]: Becoming Leader. State: Replica: c516d74195c742429b3ebd5b33d4f3fa, State: Running, Role: LEADER
I20260809 23:18:14.196084 32449 consensus_queue.cc:237] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c516d74195c742429b3ebd5b33d4f3fa" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35165 } }
I20260809 23:18:14.196549 32450 sys_catalog.cc:455] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c516d74195c742429b3ebd5b33d4f3fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c516d74195c742429b3ebd5b33d4f3fa" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35165 } } }
I20260809 23:18:14.196553 32451 sys_catalog.cc:455] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader c516d74195c742429b3ebd5b33d4f3fa. Latest consensus state: current_term: 1 leader_uuid: "c516d74195c742429b3ebd5b33d4f3fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c516d74195c742429b3ebd5b33d4f3fa" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35165 } } }
I20260809 23:18:14.196664 32450 sys_catalog.cc:458] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:14.196668 32451 sys_catalog.cc:458] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:14.196983 32468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:14.197671 32468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:14.199386 32468 catalog_manager.cc:1347] Generated new cluster ID: 0496cd7ad2fd437180a6d23b502b75e7
I20260809 23:18:14.199471 32468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:14.199671 32448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.194281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49788 (local address 127.2.76.254:35165)
0809 23:18:14.194478 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:14.194485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:14.194516 (+ 31us) server_negotiation.cc:407] Connection header received
0809 23:18:14.194617 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:14.194623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:14.194687 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:14.194832 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:14.194998 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.196684 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.197866 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.198293 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.198422 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.199023 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:14.199027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:14.199027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:14.199202 (+ 175us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:14.199246 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:14.199259 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:14.199338 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:14.199429 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 23:18:14.199512 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260809 23:18:14.207376 32468 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:14.207962 32468 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:14.211481 32468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa: Generated new TSK 0
I20260809 23:18:14.211630 32468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:14.218622 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35165
--builtin_ntp_servers=127.2.76.212:38483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:18:14.293560 32481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:14.293725 32481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:14.293745 32481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:14.294796 32481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:14.294831 32481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:14.294853 32481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:18:14.296247 32481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:38483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:14.296504 32481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:14.296859 32481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:14.302243 32492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:14.302263 32490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:14.302449 32481 server_base.cc:1034] running on GCE node
W20260809 23:18:14.302277 32496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:14.302703 32481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:14.302956 32481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:14.304132 32481 hybrid_clock.cc:648] HybridClock initialized: now 1786317494304115 us; error 31 us; skew 500 ppm
I20260809 23:18:14.305258 32481 webserver.cc:460] Webserver started at http://127.2.76.193:35721/ using document root <none> and password file <none>
I20260809 23:18:14.305491 32481 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:14.305536 32481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:14.305652 32481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:14.306535 32481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "dd5160aab2614337a9ba9ec5b82626e2"
format_stamp: "Formatted at 2026-08-09 23:18:14 on dist-test-slave-7vjv"
server_key: "fed748cb83a7282fb8228e6e22d15619"
server_key_iv: "a2ec35edc7493f7368a0c5aa71ea8cc1"
server_key_version: "encryptionkey@0"
I20260809 23:18:14.306841 32481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "dd5160aab2614337a9ba9ec5b82626e2"
format_stamp: "Formatted at 2026-08-09 23:18:14 on dist-test-slave-7vjv"
server_key: "fed748cb83a7282fb8228e6e22d15619"
server_key_iv: "a2ec35edc7493f7368a0c5aa71ea8cc1"
server_key_version: "encryptionkey@0"
I20260809 23:18:14.308152 32481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:18:14.308887 32503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:14.309036 32481 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:14.309126 32481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "dd5160aab2614337a9ba9ec5b82626e2"
format_stamp: "Formatted at 2026-08-09 23:18:14 on dist-test-slave-7vjv"
server_key: "fed748cb83a7282fb8228e6e22d15619"
server_key_iv: "a2ec35edc7493f7368a0c5aa71ea8cc1"
server_key_version: "encryptionkey@0"
I20260809 23:18:14.309190 32481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:14.326143 32481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:14.326365 32481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:14.326485 32481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:14.326659 32481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:14.326920 32481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:14.326964 32481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:14.326998 32481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:14.327028 32481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:14.332578 32481 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:46829
I20260809 23:18:14.332630 32695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:46829 every 8 connection(s)
I20260809 23:18:14.332885 32481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:18:14.333789 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 32481
I20260809 23:18:14.333887 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:18:14.334064 2355 external_mini_cluster.cc:1428] Setting key d4fd62e1a98d02059208a44408fb7c33
I20260809 23:18:14.335147 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35165
--builtin_ntp_servers=127.2.76.212:38483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 23:18:14.339025 32448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.333461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:33839 (local address 127.2.76.254:35165)
0809 23:18:14.333544 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:14.333546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:14.333749 (+ 203us) server_negotiation.cc:407] Connection header received
0809 23:18:14.334343 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:14.334344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:14.334355 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:14.334407 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:14.335337 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.336158 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.337739 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.337931 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.338002 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.338779 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:14.338781 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:14.338782 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:14.338786 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:14.338800 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:14.338804 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:14.338851 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:14.338917 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 23:18:14.338949 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 23:18:14.339226 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.333443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:35165 (local address 127.2.76.193:33839)
0809 23:18:14.333643 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:14.333670 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:14.334211 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:14.334509 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:14.334513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:14.334523 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:14.335215 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.335221 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.336318 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.336319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.337646 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.337649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.338022 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.338023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.338146 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.338649 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:18:14.338650 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:14.338675 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:18:14.338694 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:14.338851 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:14.338856 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:14.338859 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:18:14.338920 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 23:18:14.339017 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":103,"threads_started":1}
I20260809 23:18:14.339953 32697 heartbeater.cc:347] Connected to a master server at 127.2.76.254:35165
I20260809 23:18:14.340042 32697 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:14.340233 32697 heartbeater.cc:511] Master 127.2.76.254:35165 requested a full tablet report, sending...
I20260809 23:18:14.340935 32383 ts_manager.cc:194] Registered new tserver with Master: dd5160aab2614337a9ba9ec5b82626e2 (127.2.76.193:46829)
I20260809 23:18:14.341825 32383 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.76.193:33839
W20260809 23:18:14.416983 32703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:14.417156 32703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:14.417176 32703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:14.418146 32703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:14.418179 32703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:14.418202 32703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:18:14.419358 32703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:38483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:14.419507 32703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:14.419718 32703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:18:14.424252 32703 server_base.cc:1034] running on GCE node
W20260809 23:18:14.424235 32716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:14.424232 32720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:14.424222 32717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:14.424738 32703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:14.424973 32703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:14.426124 32703 hybrid_clock.cc:648] HybridClock initialized: now 1786317494426107 us; error 33 us; skew 500 ppm
I20260809 23:18:14.427313 32703 webserver.cc:460] Webserver started at http://127.2.76.194:36293/ using document root <none> and password file <none>
I20260809 23:18:14.427521 32703 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:14.427565 32703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:14.427699 32703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:14.428709 32703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "1f477833f79345fbb82ec88b2beec1dd"
format_stamp: "Formatted at 2026-08-09 23:18:14 on dist-test-slave-7vjv"
server_key: "4d1523d6bd0521b6fc629d1e4a9b3141"
server_key_iv: "38909c77f0ab52f77f3ff77878911c2a"
server_key_version: "encryptionkey@0"
I20260809 23:18:14.429037 32703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "1f477833f79345fbb82ec88b2beec1dd"
format_stamp: "Formatted at 2026-08-09 23:18:14 on dist-test-slave-7vjv"
server_key: "4d1523d6bd0521b6fc629d1e4a9b3141"
server_key_iv: "38909c77f0ab52f77f3ff77878911c2a"
server_key_version: "encryptionkey@0"
I20260809 23:18:14.430274 32703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:18:14.430922 32729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:14.431082 32703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:14.431164 32703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "1f477833f79345fbb82ec88b2beec1dd"
format_stamp: "Formatted at 2026-08-09 23:18:14 on dist-test-slave-7vjv"
server_key: "4d1523d6bd0521b6fc629d1e4a9b3141"
server_key_iv: "38909c77f0ab52f77f3ff77878911c2a"
server_key_version: "encryptionkey@0"
I20260809 23:18:14.431216 32703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:14.445977 32703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:14.446208 32703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:14.446298 32703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:14.446460 32703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:14.446691 32703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:14.446720 32703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:14.446746 32703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:14.446765 32703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:14.452600 32703 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:40075
I20260809 23:18:14.452625 458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:40075 every 8 connection(s)
I20260809 23:18:14.453096 32703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:18:14.454092 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 32703
I20260809 23:18:14.454193 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:18:14.454414 2355 external_mini_cluster.cc:1428] Setting key 673f09fc972f0b9cd648b73460b11b6b
I20260809 23:18:14.455639 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35165
--builtin_ntp_servers=127.2.76.212:38483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 23:18:14.459189 32448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.453911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:43713 (local address 127.2.76.254:35165)
0809 23:18:14.453985 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:14.453987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:14.454396 (+ 409us) server_negotiation.cc:407] Connection header received
0809 23:18:14.454975 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:14.454976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:14.455031 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:14.455121 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:14.455683 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.456396 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.457713 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.457979 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.458015 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.458947 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:14.458948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:14.458949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:14.458952 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:14.458965 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:14.458969 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:14.459023 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:14.459090 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 23:18:14.459117 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:18:14.459543 464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.453794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:35165 (local address 127.2.76.194:43713)
0809 23:18:14.454282 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:14.454323 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:14.454867 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:14.455155 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:14.455159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:14.455168 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:14.455593 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.455597 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.456480 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.456481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.457616 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.457619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.458123 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.458124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.458267 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.458810 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:18:14.458810 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:14.458839 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:18:14.458858 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:14.459026 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:14.459031 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:14.459033 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:18:14.459093 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 23:18:14.459179 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":92,"threads_started":1}
I20260809 23:18:14.460037 460 heartbeater.cc:347] Connected to a master server at 127.2.76.254:35165
I20260809 23:18:14.460139 460 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:14.460290 460 heartbeater.cc:511] Master 127.2.76.254:35165 requested a full tablet report, sending...
I20260809 23:18:14.460628 32383 ts_manager.cc:194] Registered new tserver with Master: 1f477833f79345fbb82ec88b2beec1dd (127.2.76.194:40075)
I20260809 23:18:14.460951 32383 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.76.194:43713
W20260809 23:18:14.573773 467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:14.574038 467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:14.574072 467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:14.575986 467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:14.576040 467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:14.576076 467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:18:14.578332 467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:38483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:14.578564 467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:14.578883 467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:14.583825 479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:14.583850 475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:14.583981 467 server_base.cc:1034] running on GCE node
W20260809 23:18:14.584084 472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:14.584336 467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:14.584618 467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:14.585805 467 hybrid_clock.cc:648] HybridClock initialized: now 1786317494585795 us; error 42 us; skew 500 ppm
I20260809 23:18:14.587150 467 webserver.cc:460] Webserver started at http://127.2.76.195:33457/ using document root <none> and password file <none>
I20260809 23:18:14.587417 467 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:14.587486 467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:14.587620 467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:14.588557 467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "04dc69bcf4a441989a4a01c32a4d349a"
format_stamp: "Formatted at 2026-08-09 23:18:14 on dist-test-slave-7vjv"
server_key: "62e2e6db178a5be3b08e342167a9a6a4"
server_key_iv: "72cdf1dc9581daac28e41a406ae1b49a"
server_key_version: "encryptionkey@0"
I20260809 23:18:14.588824 467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "04dc69bcf4a441989a4a01c32a4d349a"
format_stamp: "Formatted at 2026-08-09 23:18:14 on dist-test-slave-7vjv"
server_key: "62e2e6db178a5be3b08e342167a9a6a4"
server_key_iv: "72cdf1dc9581daac28e41a406ae1b49a"
server_key_version: "encryptionkey@0"
I20260809 23:18:14.589910 467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:18:14.590448 487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:14.590597 467 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:14.590684 467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "04dc69bcf4a441989a4a01c32a4d349a"
format_stamp: "Formatted at 2026-08-09 23:18:14 on dist-test-slave-7vjv"
server_key: "62e2e6db178a5be3b08e342167a9a6a4"
server_key_iv: "72cdf1dc9581daac28e41a406ae1b49a"
server_key_version: "encryptionkey@0"
I20260809 23:18:14.590745 467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:14.601893 467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:14.602135 467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:14.602270 467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:14.602478 467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:14.602813 467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:14.602860 467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:14.602895 467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:14.602917 467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:14.608349 467 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:45033
I20260809 23:18:14.608381 692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:45033 every 8 connection(s)
I20260809 23:18:14.608654 467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:18:14.612877 32448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.609280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:59707 (local address 127.2.76.254:35165)
0809 23:18:14.609339 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:14.609341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:14.609517 (+ 176us) server_negotiation.cc:407] Connection header received
0809 23:18:14.609894 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:14.609895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:14.609904 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:14.609955 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:14.610529 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.610954 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.611835 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.611996 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.612045 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.612663 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:14.612664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:14.612664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:14.612668 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:14.612680 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:14.612683 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:14.612723 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:14.612789 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 23:18:14.612811 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 23:18:14.613024 697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.609244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:35165 (local address 127.2.76.195:59707)
0809 23:18:14.609406 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:14.609439 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:14.609818 (+ 379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:14.610008 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:14.610011 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:14.610018 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:14.610451 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.610455 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.611023 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.611023 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.611730 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.611732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.612058 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.612059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.612167 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.612534 (+ 367us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:18:14.612534 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:14.612556 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:18:14.612573 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:14.612731 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:14.612734 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:14.612736 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:18:14.612782 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 23:18:14.612847 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260809 23:18:14.613469 693 heartbeater.cc:347] Connected to a master server at 127.2.76.254:35165
I20260809 23:18:14.613570 693 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:14.613566 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 467
I20260809 23:18:14.613657 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:18:14.613752 693 heartbeater.cc:511] Master 127.2.76.254:35165 requested a full tablet report, sending...
I20260809 23:18:14.613862 2355 external_mini_cluster.cc:1428] Setting key 48c8ccf13da071c99aa41e0b4d838c8e
I20260809 23:18:14.614076 32383 ts_manager.cc:194] Registered new tserver with Master: 04dc69bcf4a441989a4a01c32a4d349a (127.2.76.195:45033)
I20260809 23:18:14.614413 32383 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.76.195:59707
I20260809 23:18:14.614662 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:18:14.618269 32448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.615930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49800 (local address 127.2.76.254:35165)
0809 23:18:14.615971 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:14.615973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:14.616121 (+ 148us) server_negotiation.cc:407] Connection header received
0809 23:18:14.616209 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:14.616210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:14.616220 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:14.616251 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:14.616349 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.616732 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.617361 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.617511 (+ 150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.617555 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.618039 (+ 484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:14.618040 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:14.618041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:14.618044 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:14.618054 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:14.618057 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:14.618111 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:14.618149 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 23:18:14.618166 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 23:18:14.619472 32383 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:18:14.621054 32383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:18:14.626145 710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.622800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:45033 (local address 127.0.0.1:52436)
0809 23:18:14.622914 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:14.622949 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:14.623004 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:14.623205 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:14.623207 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:14.623210 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:14.623266 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.623268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.624350 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.624352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.625162 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.625165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.625893 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.625894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.625969 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.625980 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 23:18:14.626041 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 23:18:14.626082 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":70,"threads_started":1}
I20260809 23:18:14.626171 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.622866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52436 (local address 127.2.76.195:45033)
0809 23:18:14.622993 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:14.622996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:14.623006 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:18:14.623083 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:14.623085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:14.623098 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:14.623333 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:14.623375 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.624261 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.625244 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.625816 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.625883 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.625955 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:14.626073 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 23:18:14.626105 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":54,"threads_started":1}
I20260809 23:18:14.626748 711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.622998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:40075 (local address 127.0.0.1:54610)
0809 23:18:14.623377 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:14.623386 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:14.623435 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:14.623564 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:14.623565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:14.623567 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:14.623599 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.623600 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.624653 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.624654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.625650 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.625651 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.626527 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.626529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.626594 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.626598 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:18:14.626627 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 23:18:14.626682 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":75,"threads_started":1}
I20260809 23:18:14.627017 721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.623208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54610 (local address 127.2.76.194:40075)
0809 23:18:14.623352 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:14.623355 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:14.623425 (+ 70us) server_negotiation.cc:407] Connection header received
0809 23:18:14.623495 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:14.623496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:14.623506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:14.623554 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:14.623713 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.624575 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.625730 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.626428 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.626838 (+ 410us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.626900 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:14.626918 (+ 18us) server_negotiation.cc:299] Negotiation successful
0809 23:18:14.626946 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260809 23:18:14.626929 581 tablet_service.cc:1467] Processing CreateTablet for tablet ff4e9526b18c4daca601ee8b07c101b8 (DEFAULT_TABLE table=test-table [id=7243010a3cec4612921b0187bc01bbc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:14.627245 581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff4e9526b18c4daca601ee8b07c101b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:14.627629 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.622846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:46829 (local address 127.0.0.1:43278)
0809 23:18:14.623124 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:14.623132 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:14.623166 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:14.624366 (+ 1200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:14.624367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:14.624380 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:14.624417 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.624418 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.625513 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.625514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.626449 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.626452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.627428 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.627430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.627474 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.627477 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:18:14.627511 (+ 34us) client_negotiation.cc:240] Negotiation successful
0809 23:18:14.627579 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":74,"threads_started":1}
I20260809 23:18:14.627657 718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.623046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43278 (local address 127.2.76.193:46829)
0809 23:18:14.624202 (+ 1156us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:14.624205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:14.624215 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:18:14.624252 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:14.624254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:14.624269 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:14.624364 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:14.624466 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.625408 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.626694 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.627351 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.627426 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.627501 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:14.627527 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 23:18:14.627568 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1119,"thread_start_us":87,"threads_started":1}
I20260809 23:18:14.628217 32588 tablet_service.cc:1467] Processing CreateTablet for tablet ff4e9526b18c4daca601ee8b07c101b8 (DEFAULT_TABLE table=test-table [id=7243010a3cec4612921b0187bc01bbc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:14.628451 32588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff4e9526b18c4daca601ee8b07c101b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:14.629691 723 tablet_bootstrap.cc:492] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a: Bootstrap starting.
I20260809 23:18:14.630311 723 tablet_bootstrap.cc:654] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:14.630268 349 tablet_service.cc:1467] Processing CreateTablet for tablet ff4e9526b18c4daca601ee8b07c101b8 (DEFAULT_TABLE table=test-table [id=7243010a3cec4612921b0187bc01bbc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:14.630580 349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff4e9526b18c4daca601ee8b07c101b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:14.630591 723 log.cc:824] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:14.631260 723 tablet_bootstrap.cc:492] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a: No bootstrap required, opened a new log
I20260809 23:18:14.631242 731 tablet_bootstrap.cc:492] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2: Bootstrap starting.
I20260809 23:18:14.631325 723 ts_tablet_manager.cc:1397] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:18:14.631868 731 tablet_bootstrap.cc:654] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:14.632236 731 log.cc:824] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:14.633013 731 tablet_bootstrap.cc:492] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2: No bootstrap required, opened a new log
I20260809 23:18:14.633078 731 ts_tablet_manager.cc:1397] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:18:14.633514 733 tablet_bootstrap.cc:492] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd: Bootstrap starting.
I20260809 23:18:14.634060 733 tablet_bootstrap.cc:654] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:14.634334 733 log.cc:824] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:14.634263 723 raft_consensus.cc:348] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04dc69bcf4a441989a4a01c32a4d349a" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45033 } } peers { permanent_uuid: "1f477833f79345fbb82ec88b2beec1dd" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 40075 } } peers { permanent_uuid: "dd5160aab2614337a9ba9ec5b82626e2" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46829 } }
I20260809 23:18:14.634405 723 raft_consensus.cc:374] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:14.634449 723 raft_consensus.cc:729] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04dc69bcf4a441989a4a01c32a4d349a, State: Initialized, Role: FOLLOWER
I20260809 23:18:14.634565 723 consensus_queue.cc:260] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04dc69bcf4a441989a4a01c32a4d349a" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45033 } } peers { permanent_uuid: "1f477833f79345fbb82ec88b2beec1dd" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 40075 } } peers { permanent_uuid: "dd5160aab2614337a9ba9ec5b82626e2" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46829 } }
I20260809 23:18:14.634788 723 ts_tablet_manager.cc:1428] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:18:14.634835 693 heartbeater.cc:503] Master 127.2.76.254:35165 was elected leader, sending a full tablet report...
I20260809 23:18:14.635746 733 tablet_bootstrap.cc:492] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd: No bootstrap required, opened a new log
I20260809 23:18:14.635721 731 raft_consensus.cc:348] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04dc69bcf4a441989a4a01c32a4d349a" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45033 } } peers { permanent_uuid: "1f477833f79345fbb82ec88b2beec1dd" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 40075 } } peers { permanent_uuid: "dd5160aab2614337a9ba9ec5b82626e2" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46829 } }
I20260809 23:18:14.635816 731 raft_consensus.cc:374] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:14.635818 733 ts_tablet_manager.cc:1397] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:18:14.635835 731 raft_consensus.cc:729] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd5160aab2614337a9ba9ec5b82626e2, State: Initialized, Role: FOLLOWER
I20260809 23:18:14.635957 731 consensus_queue.cc:260] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04dc69bcf4a441989a4a01c32a4d349a" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45033 } } peers { permanent_uuid: "1f477833f79345fbb82ec88b2beec1dd" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 40075 } } peers { permanent_uuid: "dd5160aab2614337a9ba9ec5b82626e2" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46829 } }
I20260809 23:18:14.636216 731 ts_tablet_manager.cc:1428] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 23:18:14.636255 32697 heartbeater.cc:503] Master 127.2.76.254:35165 was elected leader, sending a full tablet report...
I20260809 23:18:14.637781 733 raft_consensus.cc:348] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04dc69bcf4a441989a4a01c32a4d349a" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45033 } } peers { permanent_uuid: "1f477833f79345fbb82ec88b2beec1dd" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 40075 } } peers { permanent_uuid: "dd5160aab2614337a9ba9ec5b82626e2" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46829 } }
I20260809 23:18:14.637902 733 raft_consensus.cc:374] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:14.637921 733 raft_consensus.cc:729] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f477833f79345fbb82ec88b2beec1dd, State: Initialized, Role: FOLLOWER
I20260809 23:18:14.638010 733 consensus_queue.cc:260] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04dc69bcf4a441989a4a01c32a4d349a" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45033 } } peers { permanent_uuid: "1f477833f79345fbb82ec88b2beec1dd" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 40075 } } peers { permanent_uuid: "dd5160aab2614337a9ba9ec5b82626e2" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46829 } }
I20260809 23:18:14.638212 733 ts_tablet_manager.cc:1428] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:18:14.638288 460 heartbeater.cc:503] Master 127.2.76.254:35165 was elected leader, sending a full tablet report...
I20260809 23:18:14.697280 739 raft_consensus.cc:482] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:18:14.697379 739 raft_consensus.cc:504] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04dc69bcf4a441989a4a01c32a4d349a" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45033 } } peers { permanent_uuid: "1f477833f79345fbb82ec88b2beec1dd" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 40075 } } peers { permanent_uuid: "dd5160aab2614337a9ba9ec5b82626e2" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46829 } }
I20260809 23:18:14.697731 739 leader_election.cc:288] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04dc69bcf4a441989a4a01c32a4d349a (127.2.76.195:45033), dd5160aab2614337a9ba9ec5b82626e2 (127.2.76.193:46829)
I20260809 23:18:14.700326 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.697923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:36265 (local address 127.2.76.195:45033)
0809 23:18:14.697969 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:14.697971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:14.698059 (+ 88us) server_negotiation.cc:407] Connection header received
0809 23:18:14.698138 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:14.698138 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:14.698145 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:14.698206 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:14.698345 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.698820 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.699675 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.699996 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.700074 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.700081 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:14.700238 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 23:18:14.700270 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 23:18:14.700327 464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.697849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:45033 (local address 127.2.76.194:36265)
0809 23:18:14.697988 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:14.697999 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:14.698065 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:14.698212 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:14.698213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:14.698215 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:14.698281 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.698282 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.698892 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.698893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.699593 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.699595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.700073 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.700074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.700166 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.700169 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:18:14.700232 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 23:18:14.700269 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260809 23:18:14.700580 718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.697948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:48899 (local address 127.2.76.193:46829)
0809 23:18:14.698156 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:14.698158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:14.698164 (+ 6us) server_negotiation.cc:407] Connection header received
0809 23:18:14.698184 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:14.698185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:14.698198 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:14.698246 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:14.698398 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.698907 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.699846 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.700248 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.700319 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.700326 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:14.700492 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 23:18:14.700525 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260809 23:18:14.700626 607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff4e9526b18c4daca601ee8b07c101b8" candidate_uuid: "1f477833f79345fbb82ec88b2beec1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04dc69bcf4a441989a4a01c32a4d349a" is_pre_election: true
I20260809 23:18:14.700766 607 raft_consensus.cc:2454] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f477833f79345fbb82ec88b2beec1dd in term 0.
I20260809 23:18:14.700775 741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.697864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:46829 (local address 127.2.76.194:48899)
0809 23:18:14.697995 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:14.698025 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:14.698065 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:14.698260 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:14.698261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:14.698263 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:14.698338 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.698340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.698978 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.698979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.699768 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:14.699771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.700319 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.700320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:14.700396 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.700400 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:18:14.700665 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 23:18:14.700705 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260809 23:18:14.701068 32730 leader_election.cc:302] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04dc69bcf4a441989a4a01c32a4d349a, 1f477833f79345fbb82ec88b2beec1dd; no voters:
I20260809 23:18:14.701086 32624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff4e9526b18c4daca601ee8b07c101b8" candidate_uuid: "1f477833f79345fbb82ec88b2beec1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd5160aab2614337a9ba9ec5b82626e2" is_pre_election: true
I20260809 23:18:14.701205 32624 raft_consensus.cc:2454] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f477833f79345fbb82ec88b2beec1dd in term 0.
I20260809 23:18:14.701190 739 raft_consensus.cc:2781] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:18:14.701243 739 raft_consensus.cc:482] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:18:14.701263 739 raft_consensus.cc:3037] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:14.701941 739 raft_consensus.cc:504] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04dc69bcf4a441989a4a01c32a4d349a" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45033 } } peers { permanent_uuid: "1f477833f79345fbb82ec88b2beec1dd" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 40075 } } peers { permanent_uuid: "dd5160aab2614337a9ba9ec5b82626e2" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46829 } }
I20260809 23:18:14.702065 739 leader_election.cc:288] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [CANDIDATE]: Term 1 election: Requested vote from peers 04dc69bcf4a441989a4a01c32a4d349a (127.2.76.195:45033), dd5160aab2614337a9ba9ec5b82626e2 (127.2.76.193:46829)
I20260809 23:18:14.702206 32624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff4e9526b18c4daca601ee8b07c101b8" candidate_uuid: "1f477833f79345fbb82ec88b2beec1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd5160aab2614337a9ba9ec5b82626e2"
I20260809 23:18:14.702260 32624 raft_consensus.cc:3037] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:14.703099 32624 raft_consensus.cc:2454] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f477833f79345fbb82ec88b2beec1dd in term 1.
I20260809 23:18:14.703210 607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff4e9526b18c4daca601ee8b07c101b8" candidate_uuid: "1f477833f79345fbb82ec88b2beec1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04dc69bcf4a441989a4a01c32a4d349a"
I20260809 23:18:14.703262 32732 leader_election.cc:302] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f477833f79345fbb82ec88b2beec1dd, dd5160aab2614337a9ba9ec5b82626e2; no voters:
I20260809 23:18:14.703285 607 raft_consensus.cc:3037] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:14.703366 739 raft_consensus.cc:2781] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:14.703572 739 raft_consensus.cc:686] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [term 1 LEADER]: Becoming Leader. State: Replica: 1f477833f79345fbb82ec88b2beec1dd, State: Running, Role: LEADER
I20260809 23:18:14.703657 739 consensus_queue.cc:237] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04dc69bcf4a441989a4a01c32a4d349a" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45033 } } peers { permanent_uuid: "1f477833f79345fbb82ec88b2beec1dd" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 40075 } } peers { permanent_uuid: "dd5160aab2614337a9ba9ec5b82626e2" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46829 } }
W20260809 23:18:14.703768 461 tablet.cc:2369] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:18:14.704193 607 raft_consensus.cc:2454] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f477833f79345fbb82ec88b2beec1dd in term 1.
I20260809 23:18:14.704344 32383 catalog_manager.cc:5495] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f477833f79345fbb82ec88b2beec1dd (127.2.76.194). New cstate: current_term: 1 leader_uuid: "1f477833f79345fbb82ec88b2beec1dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04dc69bcf4a441989a4a01c32a4d349a" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f477833f79345fbb82ec88b2beec1dd" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 40075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd5160aab2614337a9ba9ec5b82626e2" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46829 } health_report { overall_health: UNKNOWN } } }
I20260809 23:18:14.722549 721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.720562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54622 (local address 127.2.76.194:40075)
0809 23:18:14.720601 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:14.720603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:14.720610 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:18:14.720675 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:14.720675 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:14.720692 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:14.720750 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:14.720886 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.721260 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.721887 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:14.722058 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:14.722119 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:14.722260 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:14.722346 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:14.722416 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:14.722450 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 23:18:14.722467 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:18:14.723815 607 raft_consensus.cc:1261] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a [term 1 FOLLOWER]: Refusing update from remote peer 1f477833f79345fbb82ec88b2beec1dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:14.723810 32624 raft_consensus.cc:1261] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2 [term 1 FOLLOWER]: Refusing update from remote peer 1f477833f79345fbb82ec88b2beec1dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:14.724136 742 consensus_queue.cc:1035] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd5160aab2614337a9ba9ec5b82626e2" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 46829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:14.724243 739 consensus_queue.cc:1035] T ff4e9526b18c4daca601ee8b07c101b8 P 1f477833f79345fbb82ec88b2beec1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "04dc69bcf4a441989a4a01c32a4d349a" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:14.726325 748 mvcc.cc:205] Tried to move back new op lower bound from 7316756458386870272 to 7316756458306748416. Current Snapshot: MvccSnapshot[applied={T|T < 7316756458386870272}]
I20260809 23:18:14.726824 749 mvcc.cc:205] Tried to move back new op lower bound from 7316756458386870272 to 7316756458306748416. Current Snapshot: MvccSnapshot[applied={T|T < 7316756458386870272}]
I20260809 23:18:14.728721 32384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:18:14.728813 32384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:18:14.729532 32384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c516d74195c742429b3ebd5b33d4f3fa: Sending DeleteTablet for 3 replicas of tablet ff4e9526b18c4daca601ee8b07c101b8
I20260809 23:18:14.729861 32588 tablet_service.cc:1514] Processing DeleteTablet for tablet ff4e9526b18c4daca601ee8b07c101b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:14 UTC) from {username='slave'} at 127.0.0.1:43278
I20260809 23:18:14.729895 581 tablet_service.cc:1514] Processing DeleteTablet for tablet ff4e9526b18c4daca601ee8b07c101b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:14 UTC) from {username='slave'} at 127.0.0.1:52436
I20260809 23:18:14.730168 349 tablet_service.cc:1514] Processing DeleteTablet for tablet ff4e9526b18c4daca601ee8b07c101b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:14 UTC) from {username='slave'} at 127.0.0.1:54610
I20260809 23:18:14.730260 766 tablet_replica.cc:330] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2: stopping tablet replica
I20260809 23:18:14.730333 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 32481
I20260809 23:18:14.730355 766 raft_consensus.cc:2229] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:18:14.730446 766 raft_consensus.cc:2258] T ff4e9526b18c4daca601ee8b07c101b8 P dd5160aab2614337a9ba9ec5b82626e2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 23:18:14.738315 32356 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:46829 (error 108)
I20260809 23:18:14.738547 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 32703
I20260809 23:18:14.738740 768 tablet_replica.cc:330] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a: stopping tablet replica
I20260809 23:18:14.738828 768 raft_consensus.cc:2229] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:18:14.738912 768 raft_consensus.cc:2258] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:18:14.739101 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.738679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:46829 (local address 127.0.0.1:43282)
0809 23:18:14.739014 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:14.739066 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:46829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317}
W20260809 23:18:14.742800 32356 catalog_manager.cc:4584] TS dd5160aab2614337a9ba9ec5b82626e2 (127.2.76.193:46829): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff4e9526b18c4daca601ee8b07c101b8: Network error: Client connection negotiation failed: client connection to 127.2.76.193:46829: connect: Connection refused (error 111)
I20260809 23:18:14.744135 768 ts_tablet_manager.cc:1905] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:18:14.746147 768 ts_tablet_manager.cc:1918] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:18:14.746212 768 log.cc:1196] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/ff4e9526b18c4daca601ee8b07c101b8
I20260809 23:18:14.746407 768 ts_tablet_manager.cc:1939] T ff4e9526b18c4daca601ee8b07c101b8 P 04dc69bcf4a441989a4a01c32a4d349a: Deleting consensus metadata
I20260809 23:18:14.746786 32355 catalog_manager.cc:4841] TS 04dc69bcf4a441989a4a01c32a4d349a (127.2.76.195:45033): tablet ff4e9526b18c4daca601ee8b07c101b8 (table test-table [id=7243010a3cec4612921b0187bc01bbc1]) successfully deleted
I20260809 23:18:14.754136 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 467
I20260809 23:18:14.754153 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:14.754060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:40075 (local address 127.0.0.1:54626)
0809 23:18:14.754108 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:14.754132 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.194:40075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 23:18:14.764978 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 32330
2026-08-09T23:18:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (715 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:14 dist-test-slave-7vjv krb5kdc[776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:14 dist-test-slave-7vjv krb5kdc[776](info): set up 2 sockets
Aug 09 23:18:14 dist-test-slave-7vjv krb5kdc[776](info): commencing operation
krb5kdc: starting...
W20260809 23:18:15.918817 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:15 dist-test-slave-7vjv krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:18:16.024603 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:37321
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:37321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:16.097040 797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:16.097208 797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:16.097229 797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:16.098354 797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:16.098394 797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:16.098407 797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:16.098418 797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:16.098461 797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:16.099764 797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:37321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:37321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:16.099984 797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:16.100210 797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:16.104635 803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:16.104635 809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:16.104660 797 server_base.cc:1034] running on GCE node
W20260809 23:18:16.104655 802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:16.105046 797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:16.105275 797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:16.106395 797 hybrid_clock.cc:648] HybridClock initialized: now 1786317496106387 us; error 25 us; skew 500 ppm
Aug 09 23:18:16 dist-test-slave-7vjv krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317496, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:16.108523 797 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:18:16.108896 797 webserver.cc:460] Webserver started at http://127.2.76.254:37805/ using document root <none> and password file <none>
I20260809 23:18:16.109086 797 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:16.109139 797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:16.109259 797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:16.110100 797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "e5489a7d4e6148cabd7124cbd91c269d"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "418769b12d481613cd72bc5e362e9bdd"
server_key_iv: "358190a9cc24d0a109ed440a7f313a2b"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.110401 797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "e5489a7d4e6148cabd7124cbd91c269d"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "418769b12d481613cd72bc5e362e9bdd"
server_key_iv: "358190a9cc24d0a109ed440a7f313a2b"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.111488 797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:18:16.112067 816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.112223 797 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.112308 797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "e5489a7d4e6148cabd7124cbd91c269d"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "418769b12d481613cd72bc5e362e9bdd"
server_key_iv: "358190a9cc24d0a109ed440a7f313a2b"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.112376 797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:16.125851 797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:16.127652 797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:16.127781 797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:16.131439 797 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:37321
I20260809 23:18:16.131469 920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:37321 every 8 connection(s)
I20260809 23:18:16.131742 797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:16.132529 921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:16.134354 921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d: Bootstrap starting.
I20260809 23:18:16.134841 921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:16.135063 921 log.cc:824] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:16.135644 921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d: No bootstrap required, opened a new log
I20260809 23:18:16.137113 921 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5489a7d4e6148cabd7124cbd91c269d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37321 } }
I20260809 23:18:16.137229 921 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:16.137250 921 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5489a7d4e6148cabd7124cbd91c269d, State: Initialized, Role: FOLLOWER
I20260809 23:18:16.137404 921 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5489a7d4e6148cabd7124cbd91c269d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37321 } }
I20260809 23:18:16.137481 921 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:16.137511 921 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:16.137543 921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:16.137964 921 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5489a7d4e6148cabd7124cbd91c269d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37321 } }
I20260809 23:18:16.138038 921 leader_election.cc:302] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5489a7d4e6148cabd7124cbd91c269d; no voters:
I20260809 23:18:16.138144 921 leader_election.cc:288] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:16.138203 929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:16.138361 929 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [term 1 LEADER]: Becoming Leader. State: Replica: e5489a7d4e6148cabd7124cbd91c269d, State: Running, Role: LEADER
I20260809 23:18:16.138388 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 797
I20260809 23:18:16.138401 921 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:16.138439 929 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5489a7d4e6148cabd7124cbd91c269d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37321 } }
I20260809 23:18:16.138489 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:16.138675 2355 external_mini_cluster.cc:1428] Setting key 6bad439b07623c39e75896741c04b1f7
I20260809 23:18:16.138901 931 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5489a7d4e6148cabd7124cbd91c269d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5489a7d4e6148cabd7124cbd91c269d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37321 } } }
I20260809 23:18:16.138988 931 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:16.139320 932 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5489a7d4e6148cabd7124cbd91c269d. Latest consensus state: current_term: 1 leader_uuid: "e5489a7d4e6148cabd7124cbd91c269d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5489a7d4e6148cabd7124cbd91c269d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37321 } } }
I20260809 23:18:16.139393 932 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d [sys.catalog]: This master's current role is: LEADER
W20260809 23:18:16.140084 957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:18:16.140182 957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:18:16.140259 947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:16.140416 947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:16.141798 947 catalog_manager.cc:1347] Generated new cluster ID: f9a4d9e51bde435e905a3a7e26525f90
I20260809 23:18:16.141848 947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:18:16 dist-test-slave-7vjv krb5kdc[776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:16.148676 949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.139225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41318 (local address 127.2.76.254:37321)
0809 23:18:16.139525 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:16.139535 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:16.139788 (+ 253us) server_negotiation.cc:407] Connection header received
0809 23:18:16.139892 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:16.139898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:16.139940 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:16.140080 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:16.140802 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.141952 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.142641 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.143019 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.143122 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.145197 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:16.145201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:16.145201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:16.145211 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:16.147269 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:16.147601 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:16.147604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:16.147605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:16.147696 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:16.147889 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:16.147890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:16.147890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:16.148126 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:16.148305 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:16.148365 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 23:18:16.148492 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":56,"threads_started":1}
I20260809 23:18:16.154333 947 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:16.154881 947 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:16.161399 947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d: Generated new TSK 0
I20260809 23:18:16.161543 947 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:18:16.186975 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37321
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:16.260941 964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:16.261113 964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:16.261134 964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:16.262094 964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:16.262136 964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:16.262181 964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:18:16.263406 964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:16.263633 964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:16.263849 964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:16.268319 971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:16.268327 972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:16.268323 975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:16.268369 964 server_base.cc:1034] running on GCE node
I20260809 23:18:16.268743 964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:16.269052 964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:16.270217 964 hybrid_clock.cc:648] HybridClock initialized: now 1786317496270208 us; error 35 us; skew 500 ppm
Aug 09 23:18:16 dist-test-slave-7vjv krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317496, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:16.272570 964 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:18:16.272960 964 webserver.cc:460] Webserver started at http://127.2.76.193:43129/ using document root <none> and password file <none>
I20260809 23:18:16.273156 964 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:16.273216 964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:16.273336 964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:16.274224 964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "5686f3c3b0884262916e90535e76c07e"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "b9e341ab6b30a7d34f4105fb5fdd70ce"
server_key_iv: "cbc7a3f6aef9f3e2ad9f54d23d0b7bae"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.274556 964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "5686f3c3b0884262916e90535e76c07e"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "b9e341ab6b30a7d34f4105fb5fdd70ce"
server_key_iv: "cbc7a3f6aef9f3e2ad9f54d23d0b7bae"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.275730 964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:18:16.276451 987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.276600 964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.276692 964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "5686f3c3b0884262916e90535e76c07e"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "b9e341ab6b30a7d34f4105fb5fdd70ce"
server_key_iv: "cbc7a3f6aef9f3e2ad9f54d23d0b7bae"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.276765 964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:16.311232 964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:16.312866 964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:16.313009 964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:16.313216 964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:16.313488 964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:16.313539 964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.313577 964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:16.313611 964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.319278 964 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:38209
I20260809 23:18:16.319339 1182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:38209 every 8 connection(s)
I20260809 23:18:16.319624 964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:18:16.321201 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 964
I20260809 23:18:16.321319 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:18:16.321494 2355 external_mini_cluster.cc:1428] Setting key 93c96b81411a8df9656b2fd175f75ae4
Aug 09 23:18:16 dist-test-slave-7vjv krb5kdc[776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317496, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:16.328421 949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.320267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:35433 (local address 127.2.76.254:37321)
0809 23:18:16.320316 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:16.320318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:16.320524 (+ 206us) server_negotiation.cc:407] Connection header received
0809 23:18:16.320939 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:16.320940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:16.320954 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:16.320999 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:16.321880 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.322318 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.323161 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.323339 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.323401 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.325859 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:16.325860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:16.325860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:16.325866 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:16.327066 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:16.327530 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:16.327531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:16.327531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:16.327576 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:16.327855 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:16.327855 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:16.327856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:16.327935 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:16.327977 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:16.328306 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 23:18:16.328367 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 23:18:16.328671 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.320260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37321 (local address 127.2.76.193:35433)
0809 23:18:16.320407 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:16.320450 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:16.320842 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:16.321072 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:16.321075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:16.321638 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:16.321797 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.321801 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.322402 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.322403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.323070 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.323072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.323410 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.323411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.323523 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.324023 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:16.324033 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:16.325759 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:16.327141 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:16.327149 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:16.327152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:16.327455 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:16.327636 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:16.327637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:16.327638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:16.327779 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:16.327995 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:16.328002 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:16.328170 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 23:18:16.328283 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 23:18:16.328444 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260809 23:18:16.329123 1188 heartbeater.cc:347] Connected to a master server at 127.2.76.254:37321
I20260809 23:18:16.329217 1188 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:16.329370 1188 heartbeater.cc:511] Master 127.2.76.254:37321 requested a full tablet report, sending...
I20260809 23:18:16.329885 856 ts_manager.cc:194] Registered new tserver with Master: 5686f3c3b0884262916e90535e76c07e (127.2.76.193:38209)
I20260809 23:18:16.330405 856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:35433
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:18:16.348191 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37321
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:16.419869 1198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:16.420027 1198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:16.420045 1198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:16.421002 1198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:16.421039 1198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:16.421079 1198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:18:16.422389 1198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:16.422593 1198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:16.422780 1198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:16.427352 1212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:16.427361 1205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:16.427361 1208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:16.427356 1198 server_base.cc:1034] running on GCE node
I20260809 23:18:16.427785 1198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:16.428009 1198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:16.429144 1198 hybrid_clock.cc:648] HybridClock initialized: now 1786317496429139 us; error 40 us; skew 500 ppm
Aug 09 23:18:16 dist-test-slave-7vjv krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317496, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:16.431393 1198 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:18:16.431789 1198 webserver.cc:460] Webserver started at http://127.2.76.194:42099/ using document root <none> and password file <none>
I20260809 23:18:16.432013 1198 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:16.432060 1198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:16.432233 1198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:16.433085 1198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "0bf122c0a78174438cd5170b6f75e9de"
server_key_iv: "841c361ff66653958f85495a21d39071"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.433396 1198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "0bf122c0a78174438cd5170b6f75e9de"
server_key_iv: "841c361ff66653958f85495a21d39071"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.434514 1198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:18:16.435242 1223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.435405 1198 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:16.435487 1198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "0bf122c0a78174438cd5170b6f75e9de"
server_key_iv: "841c361ff66653958f85495a21d39071"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.435551 1198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:16.449163 1198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:16.450618 1198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:16.450755 1198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:16.450930 1198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:16.451195 1198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:16.451238 1198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.451265 1198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:16.451288 1198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.456750 1198 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:35411
I20260809 23:18:16.456782 1448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:35411 every 8 connection(s)
I20260809 23:18:16.457042 1198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:18:16 dist-test-slave-7vjv krb5kdc[776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317496, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:16.462409 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 1198
I20260809 23:18:16.462502 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:18:16.462667 2355 external_mini_cluster.cc:1428] Setting key 21db08ea8dab5e69a6ff3d21455fc3f4
I20260809 23:18:16.465276 949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.457630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:39613 (local address 127.2.76.254:37321)
0809 23:18:16.457712 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:16.457715 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:16.457857 (+ 142us) server_negotiation.cc:407] Connection header received
0809 23:18:16.458318 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:16.458319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:16.458332 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:16.458379 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:16.459181 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.459652 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.460656 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.460863 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.460933 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.462730 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:16.462731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:16.462732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:16.462738 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:16.463973 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:16.464416 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:16.464417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:16.464418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:16.464472 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:16.464706 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:16.464707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:16.464707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:16.464803 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:16.464868 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:16.465102 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 23:18:16.465196 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 23:18:16.465382 1452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.457583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37321 (local address 127.2.76.194:39613)
0809 23:18:16.457761 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:16.457789 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:16.458191 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:16.458438 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:16.458441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:16.458933 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:16.459088 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.459092 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.459756 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.459757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.460564 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.460567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.460941 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.460943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.461054 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.461431 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:16.461438 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:16.462651 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:16.464068 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:16.464075 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:16.464077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:16.464344 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:16.464552 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:16.464553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:16.464553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:16.464650 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:16.464861 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:16.464868 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:16.464984 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 23:18:16.465071 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 23:18:16.465200 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260809 23:18:16.465857 1450 heartbeater.cc:347] Connected to a master server at 127.2.76.254:37321
I20260809 23:18:16.465945 1450 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:16.466121 1450 heartbeater.cc:511] Master 127.2.76.254:37321 requested a full tablet report, sending...
I20260809 23:18:16.466420 856 ts_manager.cc:194] Registered new tserver with Master: 0b0bc55ddf0c4ff9b30b7d6b02eed794 (127.2.76.194:35411)
I20260809 23:18:16.466758 856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:39613
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:18:16.486753 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37321
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:40199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:16.561654 1459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:16.561847 1459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:16.561883 1459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:16.562875 1459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:16.562929 1459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:16.562965 1459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:18:16.564252 1459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:40199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:37321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:16.564468 1459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:16.564700 1459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:16.568885 1468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:16.568883 1474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:16.568905 1471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:16.569192 1459 server_base.cc:1034] running on GCE node
I20260809 23:18:16.569348 1459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:16.569553 1459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:16.570704 1459 hybrid_clock.cc:648] HybridClock initialized: now 1786317496570678 us; error 36 us; skew 500 ppm
Aug 09 23:18:16 dist-test-slave-7vjv krb5kdc[776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317496, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:16.572999 1459 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:18:16.573405 1459 webserver.cc:460] Webserver started at http://127.2.76.195:41775/ using document root <none> and password file <none>
I20260809 23:18:16.573632 1459 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:16.573697 1459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:16.573835 1459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:16.574769 1459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "c52b798f8e9144ab8855dcf64846e0d0"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "386f3542cc69b8d3562643ab8e2217ba"
server_key_iv: "f9d769cf7eb359f684cb1636b3d0e1a1"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.575103 1459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "c52b798f8e9144ab8855dcf64846e0d0"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "386f3542cc69b8d3562643ab8e2217ba"
server_key_iv: "f9d769cf7eb359f684cb1636b3d0e1a1"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.576309 1459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:18:16.576910 1484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.577057 1459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.577144 1459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "c52b798f8e9144ab8855dcf64846e0d0"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "386f3542cc69b8d3562643ab8e2217ba"
server_key_iv: "f9d769cf7eb359f684cb1636b3d0e1a1"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.577205 1459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:16.586833 1459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:16.588852 1459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:16.588994 1459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:16.589210 1459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:16.589473 1459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:16.589524 1459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.589560 1459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:16.589583 1459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.594790 1459 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:45635
I20260809 23:18:16.594890 1686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:45635 every 8 connection(s)
I20260809 23:18:16.595124 1459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:18:16 dist-test-slave-7vjv krb5kdc[776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317496, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:16.601167 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 1459
I20260809 23:18:16.601275 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:18:16.601466 2355 external_mini_cluster.cc:1428] Setting key 12451f68e64392f97c0c6981a4083d90
I20260809 23:18:16.603075 949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.595754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:50207 (local address 127.2.76.254:37321)
0809 23:18:16.595801 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:16.595803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:16.595984 (+ 181us) server_negotiation.cc:407] Connection header received
0809 23:18:16.596424 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:16.596425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:16.596435 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:16.596479 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:16.597304 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.597760 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.598608 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.598771 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.598816 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.600647 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:16.600648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:16.600649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:16.600654 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:16.601825 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:16.602299 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:16.602300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:16.602300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:16.602369 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:16.602560 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:16.602560 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:16.602561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:16.602660 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:16.602702 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:16.602957 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 23:18:16.603017 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:18:16.603195 1690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.595729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:37321 (local address 127.2.76.195:50207)
0809 23:18:16.595867 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:16.595896 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:16.596345 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:16.596523 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:16.596525 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:16.596969 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:16.597127 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.597130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.597836 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.597837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.598539 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.598541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.598836 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.598837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.598921 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.599362 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:16.599369 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:16.600572 (+ 1203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:16.601902 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:16.601919 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:16.601921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:16.602197 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:16.602422 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:16.602423 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:16.602423 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:16.602522 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:16.602707 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:16.602711 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:16.602840 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 23:18:16.602924 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 23:18:16.603031 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260809 23:18:16.603616 1687 heartbeater.cc:347] Connected to a master server at 127.2.76.254:37321
I20260809 23:18:16.603709 1687 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:16.603878 1687 heartbeater.cc:511] Master 127.2.76.254:37321 requested a full tablet report, sending...
I20260809 23:18:16.604254 856 ts_manager.cc:194] Registered new tserver with Master: c52b798f8e9144ab8855dcf64846e0d0 (127.2.76.195:45635)
I20260809 23:18:16.604568 856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:50207
I20260809 23:18:16.613023 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:18:16.619421 949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.614210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41324 (local address 127.2.76.254:37321)
0809 23:18:16.614241 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:16.614243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:16.614363 (+ 120us) server_negotiation.cc:407] Connection header received
0809 23:18:16.614428 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:16.614429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:16.614438 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:16.614484 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:16.614962 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.615403 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.616033 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.616249 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.616311 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.617391 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:16.617392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:16.617392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:16.617398 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:16.618320 (+ 922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:16.618700 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:16.618701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:16.618702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:16.618748 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:16.618953 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:16.618954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:16.618954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:16.619066 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:16.619166 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:16.619275 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 23:18:16.619330 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260809 23:18:16.620836 856 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:18:16.622251 856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:18:16.627669 1701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.624173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:35411 (local address 127.0.0.1:40434)
0809 23:18:16.624450 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:16.624459 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:16.624511 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:16.624684 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:16.624687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:16.624689 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:16.624746 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.624748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.625851 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.625852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.626631 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.626634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.627394 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.627394 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.627466 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.627478 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 23:18:16.627560 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 23:18:16.627608 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":28,"thread_start_us":38,"threads_started":1}
I20260809 23:18:16.627678 1700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.624237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40434 (local address 127.2.76.194:35411)
0809 23:18:16.624382 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:16.624386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:16.624516 (+ 130us) server_negotiation.cc:407] Connection header received
0809 23:18:16.624579 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:16.624581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:16.624594 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:16.624672 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:16.624803 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.625746 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.626713 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.627315 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.627390 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.627440 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:16.627575 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 23:18:16.627611 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260809 23:18:16.627950 1696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.623964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:38209 (local address 127.0.0.1:34370)
0809 23:18:16.624301 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:16.624338 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:16.624397 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:16.624746 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:16.624747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:16.624749 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:16.624789 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.624791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.625892 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.625892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.626971 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.626974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.627755 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.627757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.627825 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.627827 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:18:16.627853 (+ 26us) client_negotiation.cc:240] Negotiation successful
0809 23:18:16.627886 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":48,"thread_start_us":70,"threads_started":1}
I20260809 23:18:16.628068 1698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.624285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:45635 (local address 127.0.0.1:50636)
0809 23:18:16.624953 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:16.624962 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:16.625008 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:16.625230 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:16.625231 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:16.625232 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:16.625267 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.625267 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.626019 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.626020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.626973 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.626974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.627831 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.627831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.627888 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.627890 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:18:16.627915 (+ 25us) client_negotiation.cc:240] Negotiation successful
0809 23:18:16.627942 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":2,"thread_start_us":56,"threads_started":1}
I20260809 23:18:16.628319 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.624430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50636 (local address 127.2.76.195:45635)
0809 23:18:16.624570 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:16.624574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:16.625060 (+ 486us) server_negotiation.cc:407] Connection header received
0809 23:18:16.625095 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:16.625098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:16.625111 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:16.625171 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:16.625327 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.625928 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.627061 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.627657 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.627990 (+ 333us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.628208 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:16.628229 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 23:18:16.628259 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260809 23:18:16.628361 1697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.624125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34370 (local address 127.2.76.193:38209)
0809 23:18:16.624613 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:16.624616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:16.624625 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:18:16.624661 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:16.624663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:16.624672 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:16.624746 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:16.624860 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.625777 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.627062 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.627672 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.628133 (+ 461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.628199 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:16.628225 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 23:18:16.628268 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":76,"threads_started":1}
I20260809 23:18:16.628471 1306 tablet_service.cc:1467] Processing CreateTablet for tablet 96acbf3d8c794c06b0579ffecb15accf (DEFAULT_TABLE table=test-table [id=004227e862ab4a15966ac88f99273b1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:16.628793 1306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96acbf3d8c794c06b0579ffecb15accf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:16.628957 1566 tablet_service.cc:1467] Processing CreateTablet for tablet 96acbf3d8c794c06b0579ffecb15accf (DEFAULT_TABLE table=test-table [id=004227e862ab4a15966ac88f99273b1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:16.628973 1069 tablet_service.cc:1467] Processing CreateTablet for tablet 96acbf3d8c794c06b0579ffecb15accf (DEFAULT_TABLE table=test-table [id=004227e862ab4a15966ac88f99273b1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:16.629235 1069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96acbf3d8c794c06b0579ffecb15accf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:16.629235 1566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96acbf3d8c794c06b0579ffecb15accf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:16.631138 1708 tablet_bootstrap.cc:492] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794: Bootstrap starting.
I20260809 23:18:16.631691 1708 tablet_bootstrap.cc:654] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:16.632318 1710 tablet_bootstrap.cc:492] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e: Bootstrap starting.
I20260809 23:18:16.632383 1708 log.cc:824] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:16.632735 1709 tablet_bootstrap.cc:492] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0: Bootstrap starting.
I20260809 23:18:16.632915 1710 tablet_bootstrap.cc:654] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:16.633438 1709 tablet_bootstrap.cc:654] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:16.633527 1710 log.cc:824] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:16.633970 1709 log.cc:824] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:16.634343 1710 tablet_bootstrap.cc:492] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e: No bootstrap required, opened a new log
I20260809 23:18:16.634410 1710 ts_tablet_manager.cc:1397] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:18:16.634470 1708 tablet_bootstrap.cc:492] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794: No bootstrap required, opened a new log
I20260809 23:18:16.634539 1708 ts_tablet_manager.cc:1397] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 23:18:16.634649 1709 tablet_bootstrap.cc:492] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0: No bootstrap required, opened a new log
I20260809 23:18:16.634737 1709 ts_tablet_manager.cc:1397] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:18:16.636130 1709 raft_consensus.cc:348] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5686f3c3b0884262916e90535e76c07e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 38209 } } peers { permanent_uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 35411 } } peers { permanent_uuid: "c52b798f8e9144ab8855dcf64846e0d0" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45635 } }
I20260809 23:18:16.636276 1709 raft_consensus.cc:374] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:16.636312 1709 raft_consensus.cc:729] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c52b798f8e9144ab8855dcf64846e0d0, State: Initialized, Role: FOLLOWER
I20260809 23:18:16.636289 1708 raft_consensus.cc:348] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5686f3c3b0884262916e90535e76c07e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 38209 } } peers { permanent_uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 35411 } } peers { permanent_uuid: "c52b798f8e9144ab8855dcf64846e0d0" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45635 } }
I20260809 23:18:16.636289 1710 raft_consensus.cc:348] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5686f3c3b0884262916e90535e76c07e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 38209 } } peers { permanent_uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 35411 } } peers { permanent_uuid: "c52b798f8e9144ab8855dcf64846e0d0" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45635 } }
I20260809 23:18:16.636417 1709 consensus_queue.cc:260] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5686f3c3b0884262916e90535e76c07e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 38209 } } peers { permanent_uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 35411 } } peers { permanent_uuid: "c52b798f8e9144ab8855dcf64846e0d0" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45635 } }
I20260809 23:18:16.636453 1710 raft_consensus.cc:374] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:16.636453 1708 raft_consensus.cc:374] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:16.636500 1708 raft_consensus.cc:729] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0bc55ddf0c4ff9b30b7d6b02eed794, State: Initialized, Role: FOLLOWER
I20260809 23:18:16.636498 1710 raft_consensus.cc:729] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5686f3c3b0884262916e90535e76c07e, State: Initialized, Role: FOLLOWER
I20260809 23:18:16.636651 1709 ts_tablet_manager.cc:1428] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:18:16.636631 1708 consensus_queue.cc:260] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5686f3c3b0884262916e90535e76c07e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 38209 } } peers { permanent_uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 35411 } } peers { permanent_uuid: "c52b798f8e9144ab8855dcf64846e0d0" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45635 } }
I20260809 23:18:16.636631 1710 consensus_queue.cc:260] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5686f3c3b0884262916e90535e76c07e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 38209 } } peers { permanent_uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 35411 } } peers { permanent_uuid: "c52b798f8e9144ab8855dcf64846e0d0" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45635 } }
I20260809 23:18:16.636770 1687 heartbeater.cc:503] Master 127.2.76.254:37321 was elected leader, sending a full tablet report...
I20260809 23:18:16.636871 1710 ts_tablet_manager.cc:1428] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 23:18:16.636902 1188 heartbeater.cc:503] Master 127.2.76.254:37321 was elected leader, sending a full tablet report...
I20260809 23:18:16.637247 1708 ts_tablet_manager.cc:1428] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 23:18:16.637363 1450 heartbeater.cc:503] Master 127.2.76.254:37321 was elected leader, sending a full tablet report...
I20260809 23:18:16.649834 1719 raft_consensus.cc:482] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:18:16.649895 1719 raft_consensus.cc:504] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5686f3c3b0884262916e90535e76c07e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 38209 } } peers { permanent_uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 35411 } } peers { permanent_uuid: "c52b798f8e9144ab8855dcf64846e0d0" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45635 } }
I20260809 23:18:16.650168 1719 leader_election.cc:288] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b0bc55ddf0c4ff9b30b7d6b02eed794 (127.2.76.194:35411), c52b798f8e9144ab8855dcf64846e0d0 (127.2.76.195:45635)
I20260809 23:18:16.652642 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.650303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:35411 (local address 127.2.76.193:33331)
0809 23:18:16.650353 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:16.650363 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:16.650407 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:16.650536 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:16.650536 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:16.650538 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:16.650596 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.650597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.651171 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.651172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.651877 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.651880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.652411 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.652412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.652484 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.652487 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:18:16.652537 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 23:18:16.652580 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":3}
I20260809 23:18:16.652647 1700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.650366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:33331 (local address 127.2.76.194:35411)
0809 23:18:16.650413 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:16.650415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:16.650430 (+ 15us) server_negotiation.cc:407] Connection header received
0809 23:18:16.650449 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:16.650449 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:16.650458 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:16.650520 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:16.650658 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.651106 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.651949 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.652352 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.652399 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.652404 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:16.652555 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 23:18:16.652598 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 23:18:16.652839 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.650447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:38867 (local address 127.2.76.195:45635)
0809 23:18:16.650479 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:16.650481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:16.650618 (+ 137us) server_negotiation.cc:407] Connection header received
0809 23:18:16.650638 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:16.650639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:16.650648 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:16.650688 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:16.650815 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.651248 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.652177 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.652538 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.652592 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.652597 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:16.652758 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 23:18:16.652792 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:18:16.652940 1347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96acbf3d8c794c06b0579ffecb15accf" candidate_uuid: "5686f3c3b0884262916e90535e76c07e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794" is_pre_election: true
I20260809 23:18:16.653033 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.650403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:45635 (local address 127.2.76.193:38867)
0809 23:18:16.650535 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:16.650543 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:16.650586 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:16.650701 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:16.650701 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:16.650703 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:16.650756 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.650757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.651361 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.651362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.652105 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.652108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.652602 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.652603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.652668 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.652672 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:18:16.652909 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 23:18:16.652950 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260809 23:18:16.653052 1347 raft_consensus.cc:2454] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5686f3c3b0884262916e90535e76c07e in term 0.
I20260809 23:18:16.653261 989 leader_election.cc:302] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b0bc55ddf0c4ff9b30b7d6b02eed794, 5686f3c3b0884262916e90535e76c07e; no voters:
I20260809 23:18:16.653357 1602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96acbf3d8c794c06b0579ffecb15accf" candidate_uuid: "5686f3c3b0884262916e90535e76c07e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52b798f8e9144ab8855dcf64846e0d0" is_pre_election: true
I20260809 23:18:16.653391 1719 raft_consensus.cc:2781] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:18:16.653438 1719 raft_consensus.cc:482] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:18:16.653451 1719 raft_consensus.cc:3037] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:16.653476 1602 raft_consensus.cc:2454] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5686f3c3b0884262916e90535e76c07e in term 0.
I20260809 23:18:16.654018 1719 raft_consensus.cc:504] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5686f3c3b0884262916e90535e76c07e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 38209 } } peers { permanent_uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 35411 } } peers { permanent_uuid: "c52b798f8e9144ab8855dcf64846e0d0" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45635 } }
I20260809 23:18:16.654125 1719 leader_election.cc:288] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [CANDIDATE]: Term 1 election: Requested vote from peers 0b0bc55ddf0c4ff9b30b7d6b02eed794 (127.2.76.194:35411), c52b798f8e9144ab8855dcf64846e0d0 (127.2.76.195:45635)
I20260809 23:18:16.654273 1347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96acbf3d8c794c06b0579ffecb15accf" candidate_uuid: "5686f3c3b0884262916e90535e76c07e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794"
I20260809 23:18:16.654287 1602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96acbf3d8c794c06b0579ffecb15accf" candidate_uuid: "5686f3c3b0884262916e90535e76c07e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c52b798f8e9144ab8855dcf64846e0d0"
I20260809 23:18:16.654340 1602 raft_consensus.cc:3037] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:16.654354 1347 raft_consensus.cc:3037] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:16.654914 1602 raft_consensus.cc:2454] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5686f3c3b0884262916e90535e76c07e in term 1.
I20260809 23:18:16.654918 1347 raft_consensus.cc:2454] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5686f3c3b0884262916e90535e76c07e in term 1.
I20260809 23:18:16.655055 989 leader_election.cc:302] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b0bc55ddf0c4ff9b30b7d6b02eed794, 5686f3c3b0884262916e90535e76c07e; no voters:
I20260809 23:18:16.655138 1719 raft_consensus.cc:2781] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:16.655248 1719 raft_consensus.cc:686] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [term 1 LEADER]: Becoming Leader. State: Replica: 5686f3c3b0884262916e90535e76c07e, State: Running, Role: LEADER
I20260809 23:18:16.655323 1719 consensus_queue.cc:237] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5686f3c3b0884262916e90535e76c07e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 38209 } } peers { permanent_uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 35411 } } peers { permanent_uuid: "c52b798f8e9144ab8855dcf64846e0d0" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45635 } }
I20260809 23:18:16.655927 856 catalog_manager.cc:5495] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e reported cstate change: term changed from 0 to 1, leader changed from <none> to 5686f3c3b0884262916e90535e76c07e (127.2.76.193). New cstate: current_term: 1 leader_uuid: "5686f3c3b0884262916e90535e76c07e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5686f3c3b0884262916e90535e76c07e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 38209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 35411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c52b798f8e9144ab8855dcf64846e0d0" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45635 } health_report { overall_health: UNKNOWN } } }
I20260809 23:18:16.662334 1697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.660451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34376 (local address 127.2.76.193:38209)
0809 23:18:16.660492 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:16.660494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:16.660505 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:18:16.660529 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:16.660529 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:16.660543 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:16.660610 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:16.660702 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.661060 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.661663 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.661848 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.661907 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.662052 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:16.662129 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:16.662213 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:16.662254 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 23:18:16.662263 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 23:18:16.663589 1602 raft_consensus.cc:1261] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0 [term 1 FOLLOWER]: Refusing update from remote peer 5686f3c3b0884262916e90535e76c07e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:16.663604 1347 raft_consensus.cc:1261] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794 [term 1 FOLLOWER]: Refusing update from remote peer 5686f3c3b0884262916e90535e76c07e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:16.663866 1725 consensus_queue.cc:1035] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c52b798f8e9144ab8855dcf64846e0d0" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:16.663982 1719 consensus_queue.cc:1035] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b0bc55ddf0c4ff9b30b7d6b02eed794" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 35411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:16.665946 1731 mvcc.cc:205] Tried to move back new op lower bound from 7316756466331963392 to 7316756466300694528. Current Snapshot: MvccSnapshot[applied={T|T < 7316756466331963392}]
I20260809 23:18:16.666182 1733 mvcc.cc:205] Tried to move back new op lower bound from 7316756466331963392 to 7316756466300694528. Current Snapshot: MvccSnapshot[applied={T|T < 7316756466331963392}]
I20260809 23:18:16.666646 1732 mvcc.cc:205] Tried to move back new op lower bound from 7316756466331963392 to 7316756466300694528. Current Snapshot: MvccSnapshot[applied={T|T < 7316756466331963392 or (T in {7316756466331963392})}]
I20260809 23:18:16.668008 856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:18:16.668081 856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:18:16.668674 856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e5489a7d4e6148cabd7124cbd91c269d: Sending DeleteTablet for 3 replicas of tablet 96acbf3d8c794c06b0579ffecb15accf
I20260809 23:18:16.668933 1069 tablet_service.cc:1514] Processing DeleteTablet for tablet 96acbf3d8c794c06b0579ffecb15accf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:16 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:34370
I20260809 23:18:16.669021 1306 tablet_service.cc:1514] Processing DeleteTablet for tablet 96acbf3d8c794c06b0579ffecb15accf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:16 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:40434
I20260809 23:18:16.669052 1566 tablet_service.cc:1514] Processing DeleteTablet for tablet 96acbf3d8c794c06b0579ffecb15accf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:16 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:50636
I20260809 23:18:16.669165 1753 tablet_replica.cc:330] T 96acbf3d8c794c06b0579ffecb15accf P 5686f3c3b0884262916e90535e76c07e: stopping tablet replica
I20260809 23:18:16.669238 1755 tablet_replica.cc:330] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794: stopping tablet replica
I20260809 23:18:16.669317 1755 raft_consensus.cc:2229] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:18:16.669360 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 964
I20260809 23:18:16.669409 1755 raft_consensus.cc:2258] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:18:16.670011 1755 ts_tablet_manager.cc:1905] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:18:16.671345 1754 tablet_replica.cc:330] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0: stopping tablet replica
I20260809 23:18:16.671422 1754 raft_consensus.cc:2229] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:18:16.671492 1754 raft_consensus.cc:2258] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:18:16.672642 1755 ts_tablet_manager.cc:1918] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:18:16.672749 1755 log.cc:1196] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/96acbf3d8c794c06b0579ffecb15accf
I20260809 23:18:16.673075 1755 ts_tablet_manager.cc:1939] T 96acbf3d8c794c06b0579ffecb15accf P 0b0bc55ddf0c4ff9b30b7d6b02eed794: Deleting consensus metadata
I20260809 23:18:16.673187 1754 ts_tablet_manager.cc:1905] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:18:16.673480 823 catalog_manager.cc:4841] TS 0b0bc55ddf0c4ff9b30b7d6b02eed794 (127.2.76.194:35411): tablet 96acbf3d8c794c06b0579ffecb15accf (table test-table [id=004227e862ab4a15966ac88f99273b1e]) successfully deleted
I20260809 23:18:16.674449 1754 ts_tablet_manager.cc:1918] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:18:16.674520 1754 log.cc:1196] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/96acbf3d8c794c06b0579ffecb15accf
I20260809 23:18:16.674734 1754 ts_tablet_manager.cc:1939] T 96acbf3d8c794c06b0579ffecb15accf P c52b798f8e9144ab8855dcf64846e0d0: Deleting consensus metadata
I20260809 23:18:16.675076 823 catalog_manager.cc:4841] TS c52b798f8e9144ab8855dcf64846e0d0 (127.2.76.195:45635): tablet 96acbf3d8c794c06b0579ffecb15accf (table test-table [id=004227e862ab4a15966ac88f99273b1e]) successfully deleted
W20260809 23:18:16.681519 823 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:38209 (error 108)
I20260809 23:18:16.681764 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 1198
I20260809 23:18:16.684698 1698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.681863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:38209 (local address 127.0.0.1:34390)
0809 23:18:16.684626 (+ 2763us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:16.684672 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2740}
W20260809 23:18:16.688972 823 catalog_manager.cc:4584] TS 5686f3c3b0884262916e90535e76c07e (127.2.76.193:38209): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 96acbf3d8c794c06b0579ffecb15accf: Network error: Client connection negotiation failed: client connection to 127.2.76.193:38209: connect: Connection refused (error 111)
I20260809 23:18:16.694818 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 1459
I20260809 23:18:16.703524 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 797
2026-08-09T23:18:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1938 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T23:18:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:16Z Disabled control of system clock
I20260809 23:18:16.733342 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:35393
--webserver_interface=127.2.76.254
--webserver_port=0
--builtin_ntp_servers=127.2.76.212:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:35393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:18:16.803045 1763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:16.803294 1763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:16.803316 1763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:16.804319 1763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:16.804355 1763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:16.804366 1763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:16.804378 1763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:16.804387 1763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:16.805521 1763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:35393
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:35393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:16.805675 1763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:16.805866 1763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:16.810388 1776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:16.810467 1775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:16.810403 1779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:16.810559 1763 server_base.cc:1034] running on GCE node
I20260809 23:18:16.810817 1763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:16.811025 1763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:16.812147 1763 hybrid_clock.cc:648] HybridClock initialized: now 1786317496812120 us; error 39 us; skew 500 ppm
I20260809 23:18:16.813300 1763 webserver.cc:460] Webserver started at http://127.2.76.254:40621/ using document root <none> and password file <none>
I20260809 23:18:16.813524 1763 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:16.813565 1763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:16.813691 1763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:16.814584 1763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "6f1d19cd972f49b7bf5ef784116bd316"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "4bbd6114cd82165464bc02b9e39de644"
server_key_iv: "0e5b7494568d4ea7a45a8e9b6e846604"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.814875 1763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "6f1d19cd972f49b7bf5ef784116bd316"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "4bbd6114cd82165464bc02b9e39de644"
server_key_iv: "0e5b7494568d4ea7a45a8e9b6e846604"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.815948 1763 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:18:16.816613 1787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.816764 1763 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.816850 1763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "6f1d19cd972f49b7bf5ef784116bd316"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "4bbd6114cd82165464bc02b9e39de644"
server_key_iv: "0e5b7494568d4ea7a45a8e9b6e846604"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.816922 1763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:16.823300 1763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:16.823535 1763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:16.823658 1763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:16.827075 1763 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:35393
I20260809 23:18:16.827095 1871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:35393 every 8 connection(s)
I20260809 23:18:16.827360 1763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:16.828224 1873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:16.830087 1873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316: Bootstrap starting.
I20260809 23:18:16.830542 1873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:16.830749 1873 log.cc:824] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:16.831270 1873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316: No bootstrap required, opened a new log
I20260809 23:18:16.833031 1873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1d19cd972f49b7bf5ef784116bd316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35393 } }
I20260809 23:18:16.833176 1873 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:16.833213 1873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f1d19cd972f49b7bf5ef784116bd316, State: Initialized, Role: FOLLOWER
I20260809 23:18:16.833330 1873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1d19cd972f49b7bf5ef784116bd316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35393 } }
I20260809 23:18:16.833400 1873 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:16.833428 1873 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:16.833472 1873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:16.834105 1873 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1d19cd972f49b7bf5ef784116bd316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35393 } }
I20260809 23:18:16.834203 1873 leader_election.cc:302] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f1d19cd972f49b7bf5ef784116bd316; no voters:
I20260809 23:18:16.834421 1873 leader_election.cc:288] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:16.834467 1878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:16.834653 1873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:16.834663 1878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [term 1 LEADER]: Becoming Leader. State: Replica: 6f1d19cd972f49b7bf5ef784116bd316, State: Running, Role: LEADER
I20260809 23:18:16.834774 1878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1d19cd972f49b7bf5ef784116bd316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35393 } }
I20260809 23:18:16.835129 1880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f1d19cd972f49b7bf5ef784116bd316" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1d19cd972f49b7bf5ef784116bd316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35393 } } }
I20260809 23:18:16.835194 1882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f1d19cd972f49b7bf5ef784116bd316. Latest consensus state: current_term: 1 leader_uuid: "6f1d19cd972f49b7bf5ef784116bd316" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f1d19cd972f49b7bf5ef784116bd316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35393 } } }
I20260809 23:18:16.835230 1880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:16.835284 1882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:16.835562 1892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:16.836058 1892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:16.837272 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 1763
I20260809 23:18:16.837371 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:16.837565 2355 external_mini_cluster.cc:1428] Setting key 61974b3ee7a83c7e4e962893c9b7cc6e
I20260809 23:18:16.837697 1892 catalog_manager.cc:1347] Generated new cluster ID: 994a53d6db87420fb78a4a917339612d
I20260809 23:18:16.837760 1892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:16.852609 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.837932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53180 (local address 127.2.76.254:35393)
0809 23:18:16.838096 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:16.838103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:16.848317 (+ 10214us) server_negotiation.cc:407] Connection header received
0809 23:18:16.848418 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:16.848424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:16.848478 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:16.848589 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:16.848783 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.849899 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.850849 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.851368 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.851465 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.852026 (+ 561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:16.852029 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:16.852029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:16.852214 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:16.852247 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:16.852258 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:16.852311 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:16.852386 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 23:18:16.852447 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260809 23:18:16.860584 1892 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:16.861150 1892 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:16.865335 1892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316: Generated new TSK 0
I20260809 23:18:16.865473 1892 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:16.869131 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35393
--builtin_ntp_servers=127.2.76.212:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:18:16.960825 1923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:16.961015 1923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:16.961045 1923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:16.962024 1923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:16.962075 1923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:16.962109 1923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:18:16.963251 1923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:16.963469 1923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:16.963711 1923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:18:16.969064 1923 server_base.cc:1034] running on GCE node
W20260809 23:18:16.969031 1931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:16.969017 1929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:16.969081 1928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:16.969427 1923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:16.969666 1923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:16.970801 1923 hybrid_clock.cc:648] HybridClock initialized: now 1786317496970786 us; error 38 us; skew 500 ppm
I20260809 23:18:16.971817 1923 webserver.cc:460] Webserver started at http://127.2.76.193:35365/ using document root <none> and password file <none>
I20260809 23:18:16.972065 1923 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:16.972162 1923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:16.972293 1923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:16.973245 1923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "31932166c7b241e7baac609e13510c13"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "7d6164bab5d85f9b28dbf2a071488602"
server_key_iv: "70cec1c15dd338f5e49f4857aea9889d"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.973584 1923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "31932166c7b241e7baac609e13510c13"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "7d6164bab5d85f9b28dbf2a071488602"
server_key_iv: "70cec1c15dd338f5e49f4857aea9889d"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.974814 1923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:18:16.975441 1940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.975596 1923 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:16.975680 1923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "31932166c7b241e7baac609e13510c13"
format_stamp: "Formatted at 2026-08-09 23:18:16 on dist-test-slave-7vjv"
server_key: "7d6164bab5d85f9b28dbf2a071488602"
server_key_iv: "70cec1c15dd338f5e49f4857aea9889d"
server_key_version: "encryptionkey@0"
I20260809 23:18:16.975756 1923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:16.987387 1923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:16.987618 1923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:16.987744 1923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:16.987939 1923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:16.988296 1923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:16.988358 1923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.988389 1923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:16.988415 1923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:16.993851 1923 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:35769
I20260809 23:18:16.993886 2138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:35769 every 8 connection(s)
I20260809 23:18:16.994256 1923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:18:16.995018 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 1923
I20260809 23:18:16.995131 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:18:16.995330 2355 external_mini_cluster.cc:1428] Setting key 574b4e909ff275b102f1d88a5b62ac28
I20260809 23:18:16.996292 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35393
--builtin_ntp_servers=127.2.76.212:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 23:18:16.999747 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.994855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:51545 (local address 127.2.76.254:35393)
0809 23:18:16.994910 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:16.994913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:16.995140 (+ 227us) server_negotiation.cc:407] Connection header received
0809 23:18:16.995689 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:16.995690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:16.995703 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:16.995748 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:16.996535 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.997016 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.998396 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.998691 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.998759 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.999467 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:16.999468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:16.999468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:16.999473 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:16.999489 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:16.999494 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:16.999572 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:16.999617 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 23:18:16.999643 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 23:18:16.999830 2145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:16.994849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:35393 (local address 127.2.76.193:51545)
0809 23:18:16.995023 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:16.995060 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:16.995598 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:16.995812 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:16.995815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:16.995823 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:16.996459 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.996465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.997122 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.997123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.998296 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:16.998299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:16.998790 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:16.998791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:16.998887 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:16.999319 (+ 432us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:18:16.999319 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:16.999354 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:18:16.999373 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:16.999571 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:16.999573 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:16.999575 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:18:16.999616 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 23:18:16.999680 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":78,"threads_started":1}
I20260809 23:18:17.000347 2139 heartbeater.cc:347] Connected to a master server at 127.2.76.254:35393
I20260809 23:18:17.000443 2139 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:17.000645 2139 heartbeater.cc:511] Master 127.2.76.254:35393 requested a full tablet report, sending...
I20260809 23:18:17.001155 1820 ts_manager.cc:194] Registered new tserver with Master: 31932166c7b241e7baac609e13510c13 (127.2.76.193:35769)
I20260809 23:18:17.001698 1820 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.76.193:51545
W20260809 23:18:17.072211 2147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:17.072364 2147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:17.072384 2147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:17.073395 2147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:17.073455 2147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:17.073511 2147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:18:17.074889 2147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:17.075047 2147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:17.075294 2147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:17.079712 2157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:17.079703 2155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:17.079747 2161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:17.079950 2147 server_base.cc:1034] running on GCE node
I20260809 23:18:17.080197 2147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:17.080413 2147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:17.081542 2147 hybrid_clock.cc:648] HybridClock initialized: now 1786317497081529 us; error 35 us; skew 500 ppm
I20260809 23:18:17.085369 2147 webserver.cc:460] Webserver started at http://127.2.76.194:39239/ using document root <none> and password file <none>
I20260809 23:18:17.085561 2147 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:17.085600 2147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:17.085719 2147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:17.086570 2147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a"
format_stamp: "Formatted at 2026-08-09 23:18:17 on dist-test-slave-7vjv"
server_key: "d805b9ffcb9c54d62d213d62a1200a87"
server_key_iv: "3c73bcfeaf65045458d4d205bb3510d2"
server_key_version: "encryptionkey@0"
I20260809 23:18:17.086869 2147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a"
format_stamp: "Formatted at 2026-08-09 23:18:17 on dist-test-slave-7vjv"
server_key: "d805b9ffcb9c54d62d213d62a1200a87"
server_key_iv: "3c73bcfeaf65045458d4d205bb3510d2"
server_key_version: "encryptionkey@0"
I20260809 23:18:17.087994 2147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:18:17.088690 2170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:17.088855 2147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:17.088936 2147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a"
format_stamp: "Formatted at 2026-08-09 23:18:17 on dist-test-slave-7vjv"
server_key: "d805b9ffcb9c54d62d213d62a1200a87"
server_key_iv: "3c73bcfeaf65045458d4d205bb3510d2"
server_key_version: "encryptionkey@0"
I20260809 23:18:17.088999 2147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:17.099294 2147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:17.099514 2147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:17.099640 2147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:17.099817 2147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:17.100066 2147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:17.100111 2147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:17.100147 2147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:17.100169 2147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:17.105165 2147 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:44217
I20260809 23:18:17.105197 2405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:44217 every 8 connection(s)
I20260809 23:18:17.105474 2147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:18:17.109506 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.106047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:45843 (local address 127.2.76.254:35393)
0809 23:18:17.106121 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:17.106123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:17.106299 (+ 176us) server_negotiation.cc:407] Connection header received
0809 23:18:17.106659 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:17.106660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:17.106668 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:17.106711 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:17.107204 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.107614 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.108516 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.108682 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.108731 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.109307 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:17.109309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:17.109309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:17.109312 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:17.109324 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:17.109328 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:17.109365 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:17.109423 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 23:18:17.109443 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 23:18:17.109652 2409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.106015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:35393 (local address 127.2.76.194:45843)
0809 23:18:17.106182 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:17.106225 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:17.106589 (+ 364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:17.106752 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:17.106754 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:17.106761 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:17.107145 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:17.107148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.107691 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.107692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:17.108445 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:17.108448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.108748 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.108749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:17.108840 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.109199 (+ 359us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:18:17.109199 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:17.109222 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:18:17.109240 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:17.109371 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:17.109373 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:17.109375 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:18:17.109413 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 23:18:17.109485 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260809 23:18:17.110023 2406 heartbeater.cc:347] Connected to a master server at 127.2.76.254:35393
I20260809 23:18:17.110113 2406 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:17.110227 2406 heartbeater.cc:511] Master 127.2.76.254:35393 requested a full tablet report, sending...
I20260809 23:18:17.110509 1820 ts_manager.cc:194] Registered new tserver with Master: eb51697c1e594cf7b385ab6ce4e4cf9a (127.2.76.194:44217)
I20260809 23:18:17.110802 1820 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.76.194:45843
I20260809 23:18:17.112192 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 2147
I20260809 23:18:17.112295 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:18:17.112459 2355 external_mini_cluster.cc:1428] Setting key f22f93d5e1b67efc070b17488b0a20ad
I20260809 23:18:17.113205 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35393
--builtin_ntp_servers=127.2.76.212:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:18:17.192847 2411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:17.193065 2411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:17.193099 2411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:17.194941 2411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:17.194994 2411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:17.195034 2411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:18:17.196483 2411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:17.196696 2411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:17.197002 2411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:17.204419 2424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:17.204486 2422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:17.204653 2411 server_base.cc:1034] running on GCE node
W20260809 23:18:17.204656 2426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:17.204967 2411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:17.205206 2411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:17.206346 2411 hybrid_clock.cc:648] HybridClock initialized: now 1786317497206333 us; error 23 us; skew 500 ppm
I20260809 23:18:17.207607 2411 webserver.cc:460] Webserver started at http://127.2.76.195:35337/ using document root <none> and password file <none>
I20260809 23:18:17.207823 2411 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:17.207885 2411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:17.208007 2411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:17.209177 2411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "78d5af9b36254cfbba2ad55ae2a5d369"
format_stamp: "Formatted at 2026-08-09 23:18:17 on dist-test-slave-7vjv"
server_key: "d67ad4e04b73793b69be3e97eb8019cd"
server_key_iv: "def3777d02d2457ae37301adebba2b52"
server_key_version: "encryptionkey@0"
I20260809 23:18:17.209585 2411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "78d5af9b36254cfbba2ad55ae2a5d369"
format_stamp: "Formatted at 2026-08-09 23:18:17 on dist-test-slave-7vjv"
server_key: "d67ad4e04b73793b69be3e97eb8019cd"
server_key_iv: "def3777d02d2457ae37301adebba2b52"
server_key_version: "encryptionkey@0"
I20260809 23:18:17.210932 2411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:18:17.211757 2439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:17.211948 2411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:18:17.212033 2411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "78d5af9b36254cfbba2ad55ae2a5d369"
format_stamp: "Formatted at 2026-08-09 23:18:17 on dist-test-slave-7vjv"
server_key: "d67ad4e04b73793b69be3e97eb8019cd"
server_key_iv: "def3777d02d2457ae37301adebba2b52"
server_key_version: "encryptionkey@0"
I20260809 23:18:17.212118 2411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:17.227632 2411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:17.227864 2411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:17.227986 2411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:17.228202 2411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:17.228490 2411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:17.228538 2411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:17.228574 2411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:17.228598 2411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:17.233755 2411 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:36431
I20260809 23:18:17.233824 2638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:36431 every 8 connection(s)
I20260809 23:18:17.234118 2411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:18:17.238579 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.234807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:36859 (local address 127.2.76.254:35393)
0809 23:18:17.234848 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:17.234850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:17.235086 (+ 236us) server_negotiation.cc:407] Connection header received
0809 23:18:17.235573 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:17.235574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:17.235585 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:17.235654 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:17.236288 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.236732 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.237545 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.237707 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.237753 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.238359 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:17.238360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:17.238360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:17.238364 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:17.238376 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:17.238380 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:17.238442 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:17.238502 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 23:18:17.238524 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:18:17.238759 2643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.234804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:35393 (local address 127.2.76.195:36859)
0809 23:18:17.234985 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:17.235012 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:17.235484 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:17.235739 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:17.235742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:17.235750 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:17.236222 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:17.236226 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.236800 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.236800 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:17.237481 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:17.237483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.237771 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.237772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:17.237864 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.238249 (+ 385us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:18:17.238249 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:17.238273 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:18:17.238292 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:17.238448 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:17.238450 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:17.238452 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:18:17.238494 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 23:18:17.238559 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260809 23:18:17.239246 2639 heartbeater.cc:347] Connected to a master server at 127.2.76.254:35393
I20260809 23:18:17.239337 2639 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:17.239498 2639 heartbeater.cc:511] Master 127.2.76.254:35393 requested a full tablet report, sending...
I20260809 23:18:17.239826 1820 ts_manager.cc:194] Registered new tserver with Master: 78d5af9b36254cfbba2ad55ae2a5d369 (127.2.76.195:36431)
I20260809 23:18:17.240185 1820 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.76.195:36859
I20260809 23:18:17.241484 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 2411
I20260809 23:18:17.241590 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:18:17.241768 2355 external_mini_cluster.cc:1428] Setting key fc50feca61595311439414bdc1aa33e7
I20260809 23:18:17.242578 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:18:17.246325 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.243917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53190 (local address 127.2.76.254:35393)
0809 23:18:17.243955 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:17.243957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:17.244057 (+ 100us) server_negotiation.cc:407] Connection header received
0809 23:18:17.244194 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:17.244194 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:17.244207 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:17.244250 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:17.244376 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.244761 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.245425 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.245573 (+ 148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.245629 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.246140 (+ 511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:17.246141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:17.246142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:17.246145 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:17.246155 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:17.246158 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:17.246227 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:17.246276 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 23:18:17.246283 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:18:17.247709 1820 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:18:17.249187 1820 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:18:17.254395 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.250972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:44217 (local address 127.0.0.1:44850)
0809 23:18:17.251115 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:17.251151 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:17.251205 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:17.251602 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:17.251605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:17.251607 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:17.251672 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:17.251673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.252812 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.252813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:17.253414 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:17.253416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.254156 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.254156 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:17.254224 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.254235 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 23:18:17.254293 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 23:18:17.254335 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":6,"thread_start_us":61,"threads_started":1}
I20260809 23:18:17.254415 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.251247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44850 (local address 127.2.76.194:44217)
0809 23:18:17.251449 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:17.251452 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:17.251459 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:18:17.251507 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:17.251509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:17.251525 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:17.251600 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:17.251745 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.252732 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.253498 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.254081 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.254154 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.254231 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:17.254303 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 23:18:17.254337 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":69,"threads_started":1}
I20260809 23:18:17.254833 2654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.251162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:36431 (local address 127.0.0.1:48888)
0809 23:18:17.251424 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:17.251434 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:17.251495 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:17.251690 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:17.251691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:17.251693 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:17.251735 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:17.251737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.252598 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.252599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:17.253713 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:17.253716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.254617 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.254618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:17.254669 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.254672 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:18:17.254720 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 23:18:17.254754 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":61,"threads_started":1}
I20260809 23:18:17.254863 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.251218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48888 (local address 127.2.76.195:36431)
0809 23:18:17.251367 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:17.251371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:17.251509 (+ 138us) server_negotiation.cc:407] Connection header received
0809 23:18:17.251556 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:17.251558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:17.251570 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:17.251630 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:17.251793 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.252511 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.253823 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.254450 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.254495 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.254557 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:17.254756 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 23:18:17.254793 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
I20260809 23:18:17.255137 2272 tablet_service.cc:1467] Processing CreateTablet for tablet 31e29f4815ec42d19ecf38106897aa29 (DEFAULT_TABLE table=test-table [id=bd5c8184fbaa4227ada1cfc2a5080447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:17.255371 2272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31e29f4815ec42d19ecf38106897aa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:17.255615 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.250982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:35769 (local address 127.0.0.1:36344)
0809 23:18:17.251273 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:17.251280 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:17.251317 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:17.251611 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:17.251611 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:17.251613 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:17.251672 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:17.251675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.253902 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.253903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:17.254767 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:17.254769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.255409 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.255410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:17.255464 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.255467 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:18:17.255522 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 23:18:17.255558 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":83,"threads_started":1}
I20260809 23:18:17.255641 2657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.251073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36344 (local address 127.2.76.193:35769)
0809 23:18:17.251408 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:17.251411 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:17.251422 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:18:17.251463 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:17.251465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:17.251480 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:17.251548 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:17.252862 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.253787 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.254868 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.255308 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.255394 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.255468 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:17.255534 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 23:18:17.255567 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":75,"threads_started":1}
I20260809 23:18:17.256976 2522 tablet_service.cc:1467] Processing CreateTablet for tablet 31e29f4815ec42d19ecf38106897aa29 (DEFAULT_TABLE table=test-table [id=bd5c8184fbaa4227ada1cfc2a5080447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:17.257251 2522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31e29f4815ec42d19ecf38106897aa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:17.258006 2663 tablet_bootstrap.cc:492] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a: Bootstrap starting.
I20260809 23:18:17.258773 2663 tablet_bootstrap.cc:654] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:17.259127 2663 log.cc:824] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:17.260372 2667 tablet_bootstrap.cc:492] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369: Bootstrap starting.
I20260809 23:18:17.260480 2025 tablet_service.cc:1467] Processing CreateTablet for tablet 31e29f4815ec42d19ecf38106897aa29 (DEFAULT_TABLE table=test-table [id=bd5c8184fbaa4227ada1cfc2a5080447]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:17.260766 2025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31e29f4815ec42d19ecf38106897aa29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:17.260950 2667 tablet_bootstrap.cc:654] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:17.261205 2667 log.cc:824] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:17.261821 2667 tablet_bootstrap.cc:492] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369: No bootstrap required, opened a new log
I20260809 23:18:17.261883 2667 ts_tablet_manager.cc:1397] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:18:17.263206 2671 tablet_bootstrap.cc:492] T 31e29f4815ec42d19ecf38106897aa29 P 31932166c7b241e7baac609e13510c13: Bootstrap starting.
I20260809 23:18:17.263453 2663 tablet_bootstrap.cc:492] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a: No bootstrap required, opened a new log
I20260809 23:18:17.263520 2663 ts_tablet_manager.cc:1397] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260809 23:18:17.263798 2671 tablet_bootstrap.cc:654] T 31e29f4815ec42d19ecf38106897aa29 P 31932166c7b241e7baac609e13510c13: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:17.264199 2671 log.cc:824] T 31e29f4815ec42d19ecf38106897aa29 P 31932166c7b241e7baac609e13510c13: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:17.264147 2667 raft_consensus.cc:348] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44217 } } peers { permanent_uuid: "31932166c7b241e7baac609e13510c13" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35769 } } peers { permanent_uuid: "78d5af9b36254cfbba2ad55ae2a5d369" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36431 } }
I20260809 23:18:17.264298 2667 raft_consensus.cc:374] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:17.264360 2667 raft_consensus.cc:729] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78d5af9b36254cfbba2ad55ae2a5d369, State: Initialized, Role: FOLLOWER
I20260809 23:18:17.264496 2667 consensus_queue.cc:260] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44217 } } peers { permanent_uuid: "31932166c7b241e7baac609e13510c13" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35769 } } peers { permanent_uuid: "78d5af9b36254cfbba2ad55ae2a5d369" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36431 } }
I20260809 23:18:17.264765 2667 ts_tablet_manager.cc:1428] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:18:17.265306 2671 tablet_bootstrap.cc:492] T 31e29f4815ec42d19ecf38106897aa29 P 31932166c7b241e7baac609e13510c13: No bootstrap required, opened a new log
I20260809 23:18:17.265362 2671 ts_tablet_manager.cc:1397] T 31e29f4815ec42d19ecf38106897aa29 P 31932166c7b241e7baac609e13510c13: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:18:17.265550 2663 raft_consensus.cc:348] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44217 } } peers { permanent_uuid: "31932166c7b241e7baac609e13510c13" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35769 } } peers { permanent_uuid: "78d5af9b36254cfbba2ad55ae2a5d369" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36431 } }
I20260809 23:18:17.265671 2663 raft_consensus.cc:374] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:17.265709 2663 raft_consensus.cc:729] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb51697c1e594cf7b385ab6ce4e4cf9a, State: Initialized, Role: FOLLOWER
I20260809 23:18:17.265800 2663 consensus_queue.cc:260] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44217 } } peers { permanent_uuid: "31932166c7b241e7baac609e13510c13" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35769 } } peers { permanent_uuid: "78d5af9b36254cfbba2ad55ae2a5d369" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36431 } }
I20260809 23:18:17.266002 2663 ts_tablet_manager.cc:1428] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:18:17.266088 2406 heartbeater.cc:503] Master 127.2.76.254:35393 was elected leader, sending a full tablet report...
I20260809 23:18:17.266111 2639 heartbeater.cc:503] Master 127.2.76.254:35393 was elected leader, sending a full tablet report...
I20260809 23:18:17.267500 2671 raft_consensus.cc:348] T 31e29f4815ec42d19ecf38106897aa29 P 31932166c7b241e7baac609e13510c13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44217 } } peers { permanent_uuid: "31932166c7b241e7baac609e13510c13" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35769 } } peers { permanent_uuid: "78d5af9b36254cfbba2ad55ae2a5d369" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36431 } }
I20260809 23:18:17.267609 2671 raft_consensus.cc:374] T 31e29f4815ec42d19ecf38106897aa29 P 31932166c7b241e7baac609e13510c13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:17.267629 2671 raft_consensus.cc:729] T 31e29f4815ec42d19ecf38106897aa29 P 31932166c7b241e7baac609e13510c13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31932166c7b241e7baac609e13510c13, State: Initialized, Role: FOLLOWER
I20260809 23:18:17.267719 2671 consensus_queue.cc:260] T 31e29f4815ec42d19ecf38106897aa29 P 31932166c7b241e7baac609e13510c13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44217 } } peers { permanent_uuid: "31932166c7b241e7baac609e13510c13" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35769 } } peers { permanent_uuid: "78d5af9b36254cfbba2ad55ae2a5d369" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36431 } }
I20260809 23:18:17.267915 2671 ts_tablet_manager.cc:1428] T 31e29f4815ec42d19ecf38106897aa29 P 31932166c7b241e7baac609e13510c13: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 23:18:17.267987 2139 heartbeater.cc:503] Master 127.2.76.254:35393 was elected leader, sending a full tablet report...
W20260809 23:18:17.356048 2408 tablet.cc:2369] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:18:17.484668 2640 tablet.cc:2369] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:18:17.494865 2144 tablet.cc:2369] T 31e29f4815ec42d19ecf38106897aa29 P 31932166c7b241e7baac609e13510c13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:18:17.537828 2676 raft_consensus.cc:482] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:18:17.537942 2676 raft_consensus.cc:504] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44217 } } peers { permanent_uuid: "31932166c7b241e7baac609e13510c13" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35769 } } peers { permanent_uuid: "78d5af9b36254cfbba2ad55ae2a5d369" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36431 } }
I20260809 23:18:17.538328 2676 leader_election.cc:288] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31932166c7b241e7baac609e13510c13 (127.2.76.193:35769), 78d5af9b36254cfbba2ad55ae2a5d369 (127.2.76.195:36431)
I20260809 23:18:17.541124 2409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.538527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:35769 (local address 127.2.76.194:33353)
0809 23:18:17.538578 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:17.538590 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:17.538633 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:17.538943 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:17.538945 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:17.538946 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:17.539023 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:17.539025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.539613 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.539615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:17.540308 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:17.540310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.540875 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.540876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:17.540950 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.540953 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:18:17.541004 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 23:18:17.541034 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":2}
I20260809 23:18:17.541137 2657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.538726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:33353 (local address 127.2.76.193:35769)
0809 23:18:17.538772 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:17.538774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:17.538782 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:18:17.538811 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:17.538812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:17.538820 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:17.538880 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:17.539105 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.539527 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.540388 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.540798 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.540852 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.540858 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:17.541023 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 23:18:17.541071 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 23:18:17.541199 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.538725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:41123 (local address 127.2.76.195:36431)
0809 23:18:17.538772 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:17.538774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:17.538867 (+ 93us) server_negotiation.cc:407] Connection header received
0809 23:18:17.538899 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:17.538900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:17.538909 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:17.538957 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:17.539090 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.539522 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.540443 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.540851 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.540925 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.540932 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:17.541091 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 23:18:17.541128 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 23:18:17.541270 2678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.538635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:36431 (local address 127.2.76.194:41123)
0809 23:18:17.538795 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:17.538804 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:17.538859 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:17.538959 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:17.538959 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:17.538960 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:17.539030 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:17.539031 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.539608 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.539609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:17.540363 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:17.540365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.540936 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.540937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:17.541015 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.541018 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:18:17.541074 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 23:18:17.541108 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":113,"threads_started":1}
I20260809 23:18:17.541527 2058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e29f4815ec42d19ecf38106897aa29" candidate_uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31932166c7b241e7baac609e13510c13" is_pre_election: true
I20260809 23:18:17.541532 2556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e29f4815ec42d19ecf38106897aa29" candidate_uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d5af9b36254cfbba2ad55ae2a5d369" is_pre_election: true
I20260809 23:18:17.541683 2556 raft_consensus.cc:2454] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb51697c1e594cf7b385ab6ce4e4cf9a in term 0.
I20260809 23:18:17.541683 2058 raft_consensus.cc:2454] T 31e29f4815ec42d19ecf38106897aa29 P 31932166c7b241e7baac609e13510c13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb51697c1e594cf7b385ab6ce4e4cf9a in term 0.
I20260809 23:18:17.541913 2181 leader_election.cc:302] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31932166c7b241e7baac609e13510c13, eb51697c1e594cf7b385ab6ce4e4cf9a; no voters:
I20260809 23:18:17.542037 2676 raft_consensus.cc:2781] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:18:17.542089 2676 raft_consensus.cc:482] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:18:17.542110 2676 raft_consensus.cc:3037] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:17.542757 2676 raft_consensus.cc:504] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44217 } } peers { permanent_uuid: "31932166c7b241e7baac609e13510c13" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35769 } } peers { permanent_uuid: "78d5af9b36254cfbba2ad55ae2a5d369" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36431 } }
I20260809 23:18:17.542863 2676 leader_election.cc:288] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [CANDIDATE]: Term 1 election: Requested vote from peers 31932166c7b241e7baac609e13510c13 (127.2.76.193:35769), 78d5af9b36254cfbba2ad55ae2a5d369 (127.2.76.195:36431)
I20260809 23:18:17.543035 2556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e29f4815ec42d19ecf38106897aa29" candidate_uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78d5af9b36254cfbba2ad55ae2a5d369"
I20260809 23:18:17.543110 2058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31e29f4815ec42d19ecf38106897aa29" candidate_uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31932166c7b241e7baac609e13510c13"
I20260809 23:18:17.543123 2556 raft_consensus.cc:3037] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:17.543210 2058 raft_consensus.cc:3037] T 31e29f4815ec42d19ecf38106897aa29 P 31932166c7b241e7baac609e13510c13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:17.543934 2556 raft_consensus.cc:2454] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb51697c1e594cf7b385ab6ce4e4cf9a in term 1.
I20260809 23:18:17.544041 2058 raft_consensus.cc:2454] T 31e29f4815ec42d19ecf38106897aa29 P 31932166c7b241e7baac609e13510c13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb51697c1e594cf7b385ab6ce4e4cf9a in term 1.
I20260809 23:18:17.544152 2181 leader_election.cc:302] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78d5af9b36254cfbba2ad55ae2a5d369, eb51697c1e594cf7b385ab6ce4e4cf9a; no voters:
I20260809 23:18:17.544250 2676 raft_consensus.cc:2781] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:17.544412 2676 raft_consensus.cc:686] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [term 1 LEADER]: Becoming Leader. State: Replica: eb51697c1e594cf7b385ab6ce4e4cf9a, State: Running, Role: LEADER
I20260809 23:18:17.544509 2676 consensus_queue.cc:237] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44217 } } peers { permanent_uuid: "31932166c7b241e7baac609e13510c13" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35769 } } peers { permanent_uuid: "78d5af9b36254cfbba2ad55ae2a5d369" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36431 } }
I20260809 23:18:17.545292 1820 catalog_manager.cc:5495] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a reported cstate change: term changed from 0 to 1, leader changed from <none> to eb51697c1e594cf7b385ab6ce4e4cf9a (127.2.76.194). New cstate: current_term: 1 leader_uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb51697c1e594cf7b385ab6ce4e4cf9a" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 44217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31932166c7b241e7baac609e13510c13" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78d5af9b36254cfbba2ad55ae2a5d369" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36431 } health_report { overall_health: UNKNOWN } } }
I20260809 23:18:17.606303 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.604301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44860 (local address 127.2.76.194:44217)
0809 23:18:17.604342 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:17.604344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:17.604351 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:18:17.604407 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:17.604408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:17.604424 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:17.604477 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:17.604583 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.604972 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.605640 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:17.605805 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:17.605874 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:17.605989 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:17.606074 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:17.606140 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:17.606179 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 23:18:17.606187 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:18:17.607900 2556 raft_consensus.cc:1261] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369 [term 1 FOLLOWER]: Refusing update from remote peer eb51697c1e594cf7b385ab6ce4e4cf9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:17.607941 2058 raft_consensus.cc:1261] T 31e29f4815ec42d19ecf38106897aa29 P 31932166c7b241e7baac609e13510c13 [term 1 FOLLOWER]: Refusing update from remote peer eb51697c1e594cf7b385ab6ce4e4cf9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:17.608264 2679 consensus_queue.cc:1035] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "78d5af9b36254cfbba2ad55ae2a5d369" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 36431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:17.608382 2676 consensus_queue.cc:1035] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "31932166c7b241e7baac609e13510c13" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 35769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:17.610634 2686 mvcc.cc:205] Tried to move back new op lower bound from 7316756470199365632 to 7316756469942829056. Current Snapshot: MvccSnapshot[applied={T|T < 7316756470199365632}]
I20260809 23:18:17.611037 2689 mvcc.cc:205] Tried to move back new op lower bound from 7316756470199365632 to 7316756469942829056. Current Snapshot: MvccSnapshot[applied={T|T < 7316756470199365632}]
I20260809 23:18:17.611804 2688 mvcc.cc:205] Tried to move back new op lower bound from 7316756470199365632 to 7316756469942829056. Current Snapshot: MvccSnapshot[applied={T|T < 7316756470199365632 or (T in {7316756470199365632})}]
I20260809 23:18:17.613145 1816 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:18:17.613234 1816 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:18:17.613891 1816 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f1d19cd972f49b7bf5ef784116bd316: Sending DeleteTablet for 3 replicas of tablet 31e29f4815ec42d19ecf38106897aa29
I20260809 23:18:17.614184 2272 tablet_service.cc:1514] Processing DeleteTablet for tablet 31e29f4815ec42d19ecf38106897aa29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:17 UTC) from {username='slave'} at 127.0.0.1:44850
I20260809 23:18:17.614331 2025 tablet_service.cc:1514] Processing DeleteTablet for tablet 31e29f4815ec42d19ecf38106897aa29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:17 UTC) from {username='slave'} at 127.0.0.1:36344
I20260809 23:18:17.614383 2522 tablet_service.cc:1514] Processing DeleteTablet for tablet 31e29f4815ec42d19ecf38106897aa29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:17 UTC) from {username='slave'} at 127.0.0.1:48888
I20260809 23:18:17.614496 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 1923
I20260809 23:18:17.614624 2703 tablet_replica.cc:330] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369: stopping tablet replica
I20260809 23:18:17.614720 2700 tablet_replica.cc:330] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a: stopping tablet replica
I20260809 23:18:17.614820 2700 raft_consensus.cc:2229] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:18:17.614909 2703 raft_consensus.cc:2229] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:18:17.615085 2700 raft_consensus.cc:2258] T 31e29f4815ec42d19ecf38106897aa29 P eb51697c1e594cf7b385ab6ce4e4cf9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:18:17.615264 2703 raft_consensus.cc:2258] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:18:17.617980 2703 ts_tablet_manager.cc:1905] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:18:17.620294 2703 ts_tablet_manager.cc:1918] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:18:17.620420 2703 log.cc:1196] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/31e29f4815ec42d19ecf38106897aa29
I20260809 23:18:17.620791 2703 ts_tablet_manager.cc:1939] T 31e29f4815ec42d19ecf38106897aa29 P 78d5af9b36254cfbba2ad55ae2a5d369: Deleting consensus metadata
I20260809 23:18:17.630141 1794 catalog_manager.cc:4841] TS 78d5af9b36254cfbba2ad55ae2a5d369 (127.2.76.195:36431): tablet 31e29f4815ec42d19ecf38106897aa29 (table test-table [id=bd5c8184fbaa4227ada1cfc2a5080447]) successfully deleted
W20260809 23:18:17.630268 1794 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:35769 (error 108)
I20260809 23:18:17.630474 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 2147
I20260809 23:18:17.633567 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.630614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:35769 (local address 127.0.0.1:36346)
0809 23:18:17.633473 (+ 2859us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:17.633527 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:35769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2835}
W20260809 23:18:17.634443 1794 catalog_manager.cc:4584] TS 31932166c7b241e7baac609e13510c13 (127.2.76.193:35769): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 31e29f4815ec42d19ecf38106897aa29: Network error: Client connection negotiation failed: client connection to 127.2.76.193:35769: connect: Connection refused (error 111)
I20260809 23:18:17.645337 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:17.645214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:44217 (local address 127.0.0.1:44876)
0809 23:18:17.645275 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:17.645303 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.194:44217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 23:18:17.645429 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 2411
I20260809 23:18:17.655619 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 1763
2026-08-09T23:18:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (955 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:17 dist-test-slave-7vjv krb5kdc[2715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:17 dist-test-slave-7vjv krb5kdc[2715](info): set up 2 sockets
Aug 09 23:18:17 dist-test-slave-7vjv krb5kdc[2715](info): commencing operation
krb5kdc: starting...
W20260809 23:18:18.783355 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:18 dist-test-slave-7vjv krb5kdc[2715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:18:18.888048 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:35125
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:35125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:18.959209 2739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:18.959386 2739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:18.959405 2739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:18.960330 2739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:18.960366 2739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:18.960377 2739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:18.960388 2739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:18.960398 2739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:18.961565 2739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:35125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:35125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:18.961725 2739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:18.961862 2739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:18.965950 2750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:18.965961 2752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:18.965996 2749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:18.966043 2739 server_base.cc:1034] running on GCE node
I20260809 23:18:18.966374 2739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:18.966629 2739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:18.967896 2739 hybrid_clock.cc:648] HybridClock initialized: now 1786317498967870 us; error 32 us; skew 500 ppm
Aug 09 23:18:18 dist-test-slave-7vjv krb5kdc[2715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317498, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:18.973306 2739 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:18:18.973714 2739 webserver.cc:460] Webserver started at http://127.2.76.254:46783/ using document root <none> and password file <none>
I20260809 23:18:18.973917 2739 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:18.973975 2739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:18.974109 2739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:18.975062 2739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "552a257260184990b84c88c20b48bf83"
format_stamp: "Formatted at 2026-08-09 23:18:18 on dist-test-slave-7vjv"
server_key: "2515b0e028364389ec479f98c487d643"
server_key_iv: "77c893375b35f17595d4aca6cc06a2d3"
server_key_version: "encryptionkey@0"
I20260809 23:18:18.975406 2739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "552a257260184990b84c88c20b48bf83"
format_stamp: "Formatted at 2026-08-09 23:18:18 on dist-test-slave-7vjv"
server_key: "2515b0e028364389ec479f98c487d643"
server_key_iv: "77c893375b35f17595d4aca6cc06a2d3"
server_key_version: "encryptionkey@0"
I20260809 23:18:18.976835 2739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:18:18.977576 2767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:18.977746 2739 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:18:18.977846 2739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "552a257260184990b84c88c20b48bf83"
format_stamp: "Formatted at 2026-08-09 23:18:18 on dist-test-slave-7vjv"
server_key: "2515b0e028364389ec479f98c487d643"
server_key_iv: "77c893375b35f17595d4aca6cc06a2d3"
server_key_version: "encryptionkey@0"
I20260809 23:18:18.977916 2739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:18.999003 2739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:19.001080 2739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:19.001222 2739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:19.004920 2739 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:35125
I20260809 23:18:19.004951 2846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:35125 every 8 connection(s)
I20260809 23:18:19.005244 2739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:19.005954 2847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:19.007833 2847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83: Bootstrap starting.
I20260809 23:18:19.008380 2847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:19.008675 2847 log.cc:824] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:19.009294 2847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83: No bootstrap required, opened a new log
I20260809 23:18:19.010645 2847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552a257260184990b84c88c20b48bf83" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35125 } }
I20260809 23:18:19.010756 2847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:19.010775 2847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 552a257260184990b84c88c20b48bf83, State: Initialized, Role: FOLLOWER
I20260809 23:18:19.010859 2847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552a257260184990b84c88c20b48bf83" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35125 } }
I20260809 23:18:19.010933 2847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:19.010965 2847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:19.011015 2847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:19.011487 2847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552a257260184990b84c88c20b48bf83" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35125 } }
I20260809 23:18:19.011600 2847 leader_election.cc:302] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 552a257260184990b84c88c20b48bf83; no voters:
I20260809 23:18:19.011744 2847 leader_election.cc:288] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:19.011780 2852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:19.011968 2852 raft_consensus.cc:686] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [term 1 LEADER]: Becoming Leader. State: Replica: 552a257260184990b84c88c20b48bf83, State: Running, Role: LEADER
I20260809 23:18:19.012012 2847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:19.012063 2852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552a257260184990b84c88c20b48bf83" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35125 } }
I20260809 23:18:19.012238 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 2739
I20260809 23:18:19.012344 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:19.012427 2852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 552a257260184990b84c88c20b48bf83. Latest consensus state: current_term: 1 leader_uuid: "552a257260184990b84c88c20b48bf83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552a257260184990b84c88c20b48bf83" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35125 } } }
I20260809 23:18:19.012506 2852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:19.012547 2355 external_mini_cluster.cc:1428] Setting key 0f3f9aca021c69a3c66db5b2eeadfc69
I20260809 23:18:19.012669 2855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "552a257260184990b84c88c20b48bf83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "552a257260184990b84c88c20b48bf83" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35125 } } }
I20260809 23:18:19.012784 2855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:19.013222 2864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:19.013741 2864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:19.015848 2864 catalog_manager.cc:1347] Generated new cluster ID: 99ba72e194504d4ca84c59245b778a0d
I20260809 23:18:19.015901 2864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:18:19 dist-test-slave-7vjv krb5kdc[2715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:19.021896 2870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.013029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39760 (local address 127.2.76.254:35125)
0809 23:18:19.013197 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:19.013205 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:19.013346 (+ 141us) server_negotiation.cc:407] Connection header received
0809 23:18:19.013499 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:19.013506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:19.013573 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:19.013741 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:19.014340 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.015455 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.016310 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.016721 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.016830 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.018906 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:19.018910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:19.018910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:19.018920 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:19.020495 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:19.020833 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:19.020836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:19.020837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:19.020935 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:19.021145 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:19.021146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:19.021146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:19.021340 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:19.021493 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:19.021559 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 23:18:19.021763 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260809 23:18:19.022514 2864 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:19.023061 2864 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:19.029906 2864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83: Generated new TSK 0
I20260809 23:18:19.030045 2864 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.76.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
Entry for principal kudu/127.2.76.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab.
I20260809 23:18:19.056762 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35125
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:19.128444 2890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:19.128676 2890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:19.128710 2890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:19.129719 2890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:19.129781 2890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:19.129827 2890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:18:19.131103 2890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.193.keytab
--principal=kudu/127.2.76.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:19.131371 2890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:19.131587 2890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:19.136349 2898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:19.136360 2903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:19.136430 2890 server_base.cc:1034] running on GCE node
W20260809 23:18:19.136363 2897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:19.136744 2890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:19.136969 2890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:19.138118 2890 hybrid_clock.cc:648] HybridClock initialized: now 1786317499138114 us; error 32 us; skew 500 ppm
Aug 09 23:18:19 dist-test-slave-7vjv krb5kdc[2715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317499, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:19.140534 2890 init.cc:377] Logged in from keytab as kudu/127.2.76.193@KRBTEST.COM (short username kudu)
I20260809 23:18:19.140923 2890 webserver.cc:460] Webserver started at http://127.2.76.193:40829/ using document root <none> and password file <none>
I20260809 23:18:19.141145 2890 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:19.141207 2890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:19.141342 2890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:19.142248 2890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "01654839d0134cce807fa1e2e437b639"
format_stamp: "Formatted at 2026-08-09 23:18:19 on dist-test-slave-7vjv"
server_key: "dfe93f15cc7b6a1a9447513912cb1cf7"
server_key_iv: "60e90192e5626df1c0f693ad60501924"
server_key_version: "encryptionkey@0"
I20260809 23:18:19.142571 2890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "01654839d0134cce807fa1e2e437b639"
format_stamp: "Formatted at 2026-08-09 23:18:19 on dist-test-slave-7vjv"
server_key: "dfe93f15cc7b6a1a9447513912cb1cf7"
server_key_iv: "60e90192e5626df1c0f693ad60501924"
server_key_version: "encryptionkey@0"
I20260809 23:18:19.143688 2890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:18:19.144408 2918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:19.144567 2890 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:19.144659 2890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "01654839d0134cce807fa1e2e437b639"
format_stamp: "Formatted at 2026-08-09 23:18:19 on dist-test-slave-7vjv"
server_key: "dfe93f15cc7b6a1a9447513912cb1cf7"
server_key_iv: "60e90192e5626df1c0f693ad60501924"
server_key_version: "encryptionkey@0"
I20260809 23:18:19.144731 2890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:19.157017 2890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:19.158494 2890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:19.158649 2890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:19.158836 2890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:19.159144 2890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:19.159194 2890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:19.159226 2890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:19.159252 2890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:19.164453 2890 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:33403
I20260809 23:18:19.164534 3112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:33403 every 8 connection(s)
I20260809 23:18:19.164757 2890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:18:19 dist-test-slave-7vjv krb5kdc[2715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317499, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.193@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:19.170935 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 2890
I20260809 23:18:19.171070 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:18:19.171257 2355 external_mini_cluster.cc:1428] Setting key f5c3153fe6514030be6d7b1338e136dd
I20260809 23:18:19.172746 2870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.165396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:58861 (local address 127.2.76.254:35125)
0809 23:18:19.165443 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:19.165445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:19.165604 (+ 159us) server_negotiation.cc:407] Connection header received
0809 23:18:19.166005 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:19.166007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:19.166023 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:19.166066 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:19.166821 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.167257 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.168105 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.168301 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.168366 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.170117 (+ 1751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:19.170118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:19.170118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:19.170125 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:19.171407 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:19.171810 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:19.171812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:19.171812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:19.171880 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:19.172166 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:19.172167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:19.172168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:19.172276 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:19.172329 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:19.172623 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 23:18:19.172690 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 23:18:19.172977 3118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.165375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:35125 (local address 127.2.76.193:58861)
0809 23:18:19.165512 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:19.165540 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:19.165912 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:19.166119 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:19.166127 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:19.166580 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:19.166735 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.166738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.167341 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.167342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.167997 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.167999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.168374 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.168375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.168479 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.168837 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:19.168844 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:19.170024 (+ 1180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:19.171482 (+ 1458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:19.171488 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:19.171490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:19.171718 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:19.171951 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:19.171952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:19.171952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:19.172087 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:19.172336 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:19.172343 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:19.172482 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 23:18:19.172591 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 23:18:19.172757 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260809 23:18:19.173416 3115 heartbeater.cc:347] Connected to a master server at 127.2.76.254:35125
I20260809 23:18:19.173496 3115 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:19.173666 3115 heartbeater.cc:511] Master 127.2.76.254:35125 requested a full tablet report, sending...
I20260809 23:18:19.174157 2795 ts_manager.cc:194] Registered new tserver with Master: 01654839d0134cce807fa1e2e437b639 (127.2.76.193:33403)
I20260809 23:18:19.174721 2795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.193@KRBTEST.COM'} at 127.2.76.193:58861
WARNING: no policy specified for kudu/127.2.76.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
Entry for principal kudu/127.2.76.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab.
I20260809 23:18:19.194563 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35125
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:19.267983 3122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:19.268253 3122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:19.268278 3122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:19.269244 3122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:19.269276 3122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:19.269299 3122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:18:19.270474 3122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.194.keytab
--principal=kudu/127.2.76.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:19.270623 3122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:19.270768 3122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:19.275169 3134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:19.275156 3144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:19.275200 3122 server_base.cc:1034] running on GCE node
W20260809 23:18:19.275156 3137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:19.275557 3122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:19.275768 3122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:19.276945 3122 hybrid_clock.cc:648] HybridClock initialized: now 1786317499276925 us; error 33 us; skew 500 ppm
Aug 09 23:18:19 dist-test-slave-7vjv krb5kdc[2715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317499, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:19.279075 3122 init.cc:377] Logged in from keytab as kudu/127.2.76.194@KRBTEST.COM (short username kudu)
I20260809 23:18:19.279481 3122 webserver.cc:460] Webserver started at http://127.2.76.194:41873/ using document root <none> and password file <none>
I20260809 23:18:19.279681 3122 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:19.279739 3122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:19.279866 3122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:19.280786 3122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "f1c3e63b1e5445b89c711a532a7fcc74"
format_stamp: "Formatted at 2026-08-09 23:18:19 on dist-test-slave-7vjv"
server_key: "4a67f0e0ce07e5f1691e688ea871a28f"
server_key_iv: "760b7c62165e8979234b3d0c3e7bc8b2"
server_key_version: "encryptionkey@0"
I20260809 23:18:19.281113 3122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "f1c3e63b1e5445b89c711a532a7fcc74"
format_stamp: "Formatted at 2026-08-09 23:18:19 on dist-test-slave-7vjv"
server_key: "4a67f0e0ce07e5f1691e688ea871a28f"
server_key_iv: "760b7c62165e8979234b3d0c3e7bc8b2"
server_key_version: "encryptionkey@0"
I20260809 23:18:19.282346 3122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:18:19.282980 3161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:19.283133 3122 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:18:19.283215 3122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "f1c3e63b1e5445b89c711a532a7fcc74"
format_stamp: "Formatted at 2026-08-09 23:18:19 on dist-test-slave-7vjv"
server_key: "4a67f0e0ce07e5f1691e688ea871a28f"
server_key_iv: "760b7c62165e8979234b3d0c3e7bc8b2"
server_key_version: "encryptionkey@0"
I20260809 23:18:19.283284 3122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:19.295814 3122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:19.297405 3122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:19.297542 3122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:19.297729 3122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:19.297986 3122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:19.298053 3122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:19.298082 3122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:19.298106 3122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:19.303335 3122 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:36569
I20260809 23:18:19.303377 3353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:36569 every 8 connection(s)
I20260809 23:18:19.303645 3122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:18:19.308698 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 3122
I20260809 23:18:19.308784 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
Aug 09 23:18:19 dist-test-slave-7vjv krb5kdc[2715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317499, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.194@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:19.308998 2355 external_mini_cluster.cc:1428] Setting key 604ddacae42dcfdb433442a4825b88a5
I20260809 23:18:19.312162 2870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.304313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:55481 (local address 127.2.76.254:35125)
0809 23:18:19.304357 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:19.304358 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:19.304519 (+ 161us) server_negotiation.cc:407] Connection header received
0809 23:18:19.304891 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:19.304892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:19.304902 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:19.304948 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:19.305812 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.306272 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.307136 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.307308 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.307355 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.309373 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:19.309374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:19.309374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:19.309385 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:19.310747 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:19.311192 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:19.311193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:19.311193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:19.311242 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:19.311523 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:19.311524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:19.311524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:19.311661 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:19.311732 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:19.311937 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 23:18:19.312019 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:18:19.312199 3357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.304268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:35125 (local address 127.2.76.194:55481)
0809 23:18:19.304405 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:19.304441 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:19.304818 (+ 377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:19.304999 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:19.305008 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:19.305581 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:19.305742 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.305746 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.306341 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.306341 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.307051 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.307052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.307373 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.307374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.307460 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.307821 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:19.307828 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:19.309276 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:19.310825 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:19.310832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:19.310834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:19.311127 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:19.311300 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:19.311300 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:19.311301 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:19.311440 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:19.311726 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:19.311730 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:19.311824 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 23:18:19.311897 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:18:19.311999 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260809 23:18:19.312703 3354 heartbeater.cc:347] Connected to a master server at 127.2.76.254:35125
I20260809 23:18:19.312781 3354 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:19.312943 3354 heartbeater.cc:511] Master 127.2.76.254:35125 requested a full tablet report, sending...
I20260809 23:18:19.313306 2795 ts_manager.cc:194] Registered new tserver with Master: f1c3e63b1e5445b89c711a532a7fcc74 (127.2.76.194:36569)
I20260809 23:18:19.313767 2795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.194@KRBTEST.COM'} at 127.2.76.194:55481
WARNING: no policy specified for kudu/127.2.76.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
Entry for principal kudu/127.2.76.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab.
I20260809 23:18:19.332876 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35125
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:34545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:19.406947 3362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:19.407202 3362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:19.407230 3362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:19.408231 3362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:19.408265 3362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:19.408288 3362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:18:19.409629 3362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:34545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.195.keytab
--principal=kudu/127.2.76.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:19.409869 3362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:19.410063 3362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:19.414633 3380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:19.414695 3374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:19.414786 3362 server_base.cc:1034] running on GCE node
W20260809 23:18:19.414649 3372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:19.415135 3362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:19.415378 3362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:19.416517 3362 hybrid_clock.cc:648] HybridClock initialized: now 1786317499416505 us; error 26 us; skew 500 ppm
Aug 09 23:18:19 dist-test-slave-7vjv krb5kdc[2715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317499, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:19.418661 3362 init.cc:377] Logged in from keytab as kudu/127.2.76.195@KRBTEST.COM (short username kudu)
I20260809 23:18:19.419070 3362 webserver.cc:460] Webserver started at http://127.2.76.195:34669/ using document root <none> and password file <none>
I20260809 23:18:19.419286 3362 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:19.419323 3362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:19.419443 3362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:19.420338 3362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "85e942593c7d43949b740bc98d5a0758"
format_stamp: "Formatted at 2026-08-09 23:18:19 on dist-test-slave-7vjv"
server_key: "905a367f19e2e69e5db291a984d27b22"
server_key_iv: "e1168270ed4062f6bf0d7074b02edaa2"
server_key_version: "encryptionkey@0"
I20260809 23:18:19.420637 3362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "85e942593c7d43949b740bc98d5a0758"
format_stamp: "Formatted at 2026-08-09 23:18:19 on dist-test-slave-7vjv"
server_key: "905a367f19e2e69e5db291a984d27b22"
server_key_iv: "e1168270ed4062f6bf0d7074b02edaa2"
server_key_version: "encryptionkey@0"
I20260809 23:18:19.421818 3362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:18:19.422396 3394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:19.422544 3362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:19.422627 3362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "85e942593c7d43949b740bc98d5a0758"
format_stamp: "Formatted at 2026-08-09 23:18:19 on dist-test-slave-7vjv"
server_key: "905a367f19e2e69e5db291a984d27b22"
server_key_iv: "e1168270ed4062f6bf0d7074b02edaa2"
server_key_version: "encryptionkey@0"
I20260809 23:18:19.422694 3362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:19.432787 3362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:19.434201 3362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:19.434335 3362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:19.434520 3362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:19.434773 3362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:19.434818 3362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:19.434844 3362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:19.434868 3362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:19.440222 3362 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:45057
I20260809 23:18:19.440258 3592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:45057 every 8 connection(s)
I20260809 23:18:19.440541 3362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:18:19 dist-test-slave-7vjv krb5kdc[2715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317499, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.195@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:19.446949 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 3362
I20260809 23:18:19.447103 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:18:19.447316 2355 external_mini_cluster.cc:1428] Setting key ba701c5533c8ccb47798bb83aef85108
I20260809 23:18:19.448658 2870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.441167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:58215 (local address 127.2.76.254:35125)
0809 23:18:19.441234 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:19.441236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:19.441379 (+ 143us) server_negotiation.cc:407] Connection header received
0809 23:18:19.441774 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:19.441775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:19.441785 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:19.441836 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:19.442659 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.443147 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.443979 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.444206 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.444250 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.445993 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:19.445994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:19.445994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:19.445999 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:19.447135 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:19.447625 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:19.447627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:19.447627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:19.447689 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:19.447975 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:19.447976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:19.447976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:19.448146 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:19.448206 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:19.448485 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 23:18:19.448573 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 23:18:19.448803 3596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.441125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:35125 (local address 127.2.76.195:58215)
0809 23:18:19.441279 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:19.441309 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:19.441684 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:19.441881 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:19.441895 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:19.442408 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:19.442588 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.442593 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.443217 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.443218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.443905 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.443906 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.444259 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.444260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.444351 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.444767 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:19.444774 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:19.445917 (+ 1143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:19.447227 (+ 1310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:19.447235 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:19.447237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:19.447541 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:19.447762 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:19.447763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:19.447763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:19.447907 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:19.448208 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:19.448216 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:19.448365 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 23:18:19.448461 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 23:18:19.448604 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260809 23:18:19.449162 3593 heartbeater.cc:347] Connected to a master server at 127.2.76.254:35125
I20260809 23:18:19.449251 3593 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:19.449401 3593 heartbeater.cc:511] Master 127.2.76.254:35125 requested a full tablet report, sending...
I20260809 23:18:19.449726 2795 ts_manager.cc:194] Registered new tserver with Master: 85e942593c7d43949b740bc98d5a0758 (127.2.76.195:45057)
I20260809 23:18:19.450045 2795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.76.195@KRBTEST.COM'} at 127.2.76.195:58215
I20260809 23:18:19.459087 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:18:19.465003 2870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.460362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34724 (local address 127.2.76.254:35125)
0809 23:18:19.460399 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:19.460401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:19.460474 (+ 73us) server_negotiation.cc:407] Connection header received
0809 23:18:19.460525 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:19.460525 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:19.460532 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:19.460571 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:19.461028 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.461392 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.461994 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.462136 (+ 142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.462176 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.463040 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:19.463041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:19.463041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:19.463047 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:19.464232 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:19.464483 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:19.464484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:19.464484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:19.464533 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:19.464657 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:19.464658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:19.464658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:19.464755 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:19.464813 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:19.464885 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 23:18:19.464933 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 23:18:19.466368 2795 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:18:19.467738 2795 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:18:19.473078 3614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.469653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:36569 (local address 127.0.0.1:38398)
0809 23:18:19.469920 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:19.469954 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:19.470027 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:19.470180 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:19.470182 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:19.470184 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:19.470239 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.470241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.471153 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.471154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.471969 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.471971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.472769 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.472770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.472851 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.472864 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 23:18:19.472952 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 23:18:19.473005 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":74,"threads_started":1}
I20260809 23:18:19.473086 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.469754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.2.76.194:36569)
0809 23:18:19.470042 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:19.470046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:19.470056 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:18:19.470095 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:19.470096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:19.470109 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:19.470171 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:19.470292 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.471062 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.472055 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.472692 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.472767 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.472820 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:19.472971 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 23:18:19.473018 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":75,"threads_started":1}
I20260809 23:18:19.473668 3245 tablet_service.cc:1467] Processing CreateTablet for tablet 271257248fec40efaa31f17b24cb99a6 (DEFAULT_TABLE table=test-table [id=c5646ca889dd4c03a942ce786763b121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:19.474045 3245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 271257248fec40efaa31f17b24cb99a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:19.474054 3619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.469863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35140 (local address 127.2.76.195:45057)
0809 23:18:19.470012 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:19.470016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:19.470382 (+ 366us) server_negotiation.cc:407] Connection header received
0809 23:18:19.470420 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:19.470422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:19.470434 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:19.470494 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:19.470668 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.471640 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.472836 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.473700 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.473771 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.473884 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:19.473942 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 23:18:19.473979 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260809 23:18:19.474071 3616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.469660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:45057 (local address 127.0.0.1:35140)
0809 23:18:19.470284 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:19.470291 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:19.470331 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:19.470560 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:19.470561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:19.470563 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:19.470597 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.470598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.471750 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.471751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.472748 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.472751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.473791 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.473792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.473849 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.473853 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:18:19.473927 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 23:18:19.473967 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":3,"thread_start_us":103,"threads_started":1}
I20260809 23:18:19.474247 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.469660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:33403 (local address 127.0.0.1:56168)
0809 23:18:19.470128 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:19.470136 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:19.470209 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:19.470372 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:19.470373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:19.470375 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:19.470403 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.470404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.472129 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.472129 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.473090 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.473093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.474017 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.474018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.474090 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.474094 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:18:19.474145 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 23:18:19.474189 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":1,"thread_start_us":87,"threads_started":1}
I20260809 23:18:19.474542 3623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.469893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56168 (local address 127.2.76.193:33403)
0809 23:18:19.470088 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:19.470090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:19.470202 (+ 112us) server_negotiation.cc:407] Connection header received
0809 23:18:19.470282 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:19.470284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:19.470300 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:19.470539 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:19.470616 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.471640 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.473184 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.473848 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.473924 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.473973 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:19.474419 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 23:18:19.474470 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":87,"threads_started":1}
I20260809 23:18:19.474962 3484 tablet_service.cc:1467] Processing CreateTablet for tablet 271257248fec40efaa31f17b24cb99a6 (DEFAULT_TABLE table=test-table [id=c5646ca889dd4c03a942ce786763b121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:19.475060 2993 tablet_service.cc:1467] Processing CreateTablet for tablet 271257248fec40efaa31f17b24cb99a6 (DEFAULT_TABLE table=test-table [id=c5646ca889dd4c03a942ce786763b121]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:19.475282 3484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 271257248fec40efaa31f17b24cb99a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:19.475375 2993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 271257248fec40efaa31f17b24cb99a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:19.477385 3624 tablet_bootstrap.cc:492] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74: Bootstrap starting.
I20260809 23:18:19.478097 3624 tablet_bootstrap.cc:654] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:19.478078 3626 tablet_bootstrap.cc:492] T 271257248fec40efaa31f17b24cb99a6 P 01654839d0134cce807fa1e2e437b639: Bootstrap starting.
I20260809 23:18:19.478418 3627 tablet_bootstrap.cc:492] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758: Bootstrap starting.
I20260809 23:18:19.478670 3624 log.cc:824] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:19.478718 3626 tablet_bootstrap.cc:654] T 271257248fec40efaa31f17b24cb99a6 P 01654839d0134cce807fa1e2e437b639: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:19.479070 3627 tablet_bootstrap.cc:654] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:19.479291 3626 log.cc:824] T 271257248fec40efaa31f17b24cb99a6 P 01654839d0134cce807fa1e2e437b639: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:19.479502 3624 tablet_bootstrap.cc:492] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74: No bootstrap required, opened a new log
I20260809 23:18:19.479588 3624 ts_tablet_manager.cc:1397] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:18:19.479635 3627 log.cc:824] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:19.480130 3626 tablet_bootstrap.cc:492] T 271257248fec40efaa31f17b24cb99a6 P 01654839d0134cce807fa1e2e437b639: No bootstrap required, opened a new log
I20260809 23:18:19.480252 3626 ts_tablet_manager.cc:1397] T 271257248fec40efaa31f17b24cb99a6 P 01654839d0134cce807fa1e2e437b639: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:18:19.480434 3627 tablet_bootstrap.cc:492] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758: No bootstrap required, opened a new log
I20260809 23:18:19.480499 3627 ts_tablet_manager.cc:1397] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:18:19.481112 3624 raft_consensus.cc:348] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c3e63b1e5445b89c711a532a7fcc74" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36569 } } peers { permanent_uuid: "85e942593c7d43949b740bc98d5a0758" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45057 } } peers { permanent_uuid: "01654839d0134cce807fa1e2e437b639" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33403 } }
I20260809 23:18:19.481267 3624 raft_consensus.cc:374] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:19.481310 3624 raft_consensus.cc:729] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1c3e63b1e5445b89c711a532a7fcc74, State: Initialized, Role: FOLLOWER
I20260809 23:18:19.481421 3624 consensus_queue.cc:260] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c3e63b1e5445b89c711a532a7fcc74" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36569 } } peers { permanent_uuid: "85e942593c7d43949b740bc98d5a0758" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45057 } } peers { permanent_uuid: "01654839d0134cce807fa1e2e437b639" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33403 } }
I20260809 23:18:19.481680 3624 ts_tablet_manager.cc:1428] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 23:18:19.481813 3354 heartbeater.cc:503] Master 127.2.76.254:35125 was elected leader, sending a full tablet report...
I20260809 23:18:19.482239 3626 raft_consensus.cc:348] T 271257248fec40efaa31f17b24cb99a6 P 01654839d0134cce807fa1e2e437b639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c3e63b1e5445b89c711a532a7fcc74" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36569 } } peers { permanent_uuid: "85e942593c7d43949b740bc98d5a0758" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45057 } } peers { permanent_uuid: "01654839d0134cce807fa1e2e437b639" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33403 } }
I20260809 23:18:19.482396 3626 raft_consensus.cc:374] T 271257248fec40efaa31f17b24cb99a6 P 01654839d0134cce807fa1e2e437b639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:19.482427 3626 raft_consensus.cc:729] T 271257248fec40efaa31f17b24cb99a6 P 01654839d0134cce807fa1e2e437b639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01654839d0134cce807fa1e2e437b639, State: Initialized, Role: FOLLOWER
I20260809 23:18:19.482564 3626 consensus_queue.cc:260] T 271257248fec40efaa31f17b24cb99a6 P 01654839d0134cce807fa1e2e437b639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c3e63b1e5445b89c711a532a7fcc74" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36569 } } peers { permanent_uuid: "85e942593c7d43949b740bc98d5a0758" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45057 } } peers { permanent_uuid: "01654839d0134cce807fa1e2e437b639" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33403 } }
I20260809 23:18:19.482662 3627 raft_consensus.cc:348] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c3e63b1e5445b89c711a532a7fcc74" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36569 } } peers { permanent_uuid: "85e942593c7d43949b740bc98d5a0758" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45057 } } peers { permanent_uuid: "01654839d0134cce807fa1e2e437b639" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33403 } }
I20260809 23:18:19.482844 3626 ts_tablet_manager.cc:1428] T 271257248fec40efaa31f17b24cb99a6 P 01654839d0134cce807fa1e2e437b639: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 23:18:19.482839 3627 raft_consensus.cc:374] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:19.482882 3627 raft_consensus.cc:729] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e942593c7d43949b740bc98d5a0758, State: Initialized, Role: FOLLOWER
I20260809 23:18:19.482903 3115 heartbeater.cc:503] Master 127.2.76.254:35125 was elected leader, sending a full tablet report...
I20260809 23:18:19.482976 3627 consensus_queue.cc:260] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c3e63b1e5445b89c711a532a7fcc74" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36569 } } peers { permanent_uuid: "85e942593c7d43949b740bc98d5a0758" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45057 } } peers { permanent_uuid: "01654839d0134cce807fa1e2e437b639" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33403 } }
I20260809 23:18:19.483227 3627 ts_tablet_manager.cc:1428] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:18:19.483251 3593 heartbeater.cc:503] Master 127.2.76.254:35125 was elected leader, sending a full tablet report...
W20260809 23:18:19.554229 3356 tablet.cc:2369] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:18:19.665444 3117 tablet.cc:2369] T 271257248fec40efaa31f17b24cb99a6 P 01654839d0134cce807fa1e2e437b639: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:18:19.687201 3634 raft_consensus.cc:482] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:18:19.687418 3634 raft_consensus.cc:504] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c3e63b1e5445b89c711a532a7fcc74" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36569 } } peers { permanent_uuid: "85e942593c7d43949b740bc98d5a0758" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45057 } } peers { permanent_uuid: "01654839d0134cce807fa1e2e437b639" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33403 } }
I20260809 23:18:19.687788 3634 leader_election.cc:288] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85e942593c7d43949b740bc98d5a0758 (127.2.76.195:45057), 01654839d0134cce807fa1e2e437b639 (127.2.76.193:33403)
I20260809 23:18:19.690615 3623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.688099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:35695 (local address 127.2.76.193:33403)
0809 23:18:19.688196 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:19.688198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:19.688291 (+ 93us) server_negotiation.cc:407] Connection header received
0809 23:18:19.688359 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:19.688360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:19.688376 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:19.688437 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:19.688589 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.689048 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.689977 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.690296 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.690369 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.690377 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:19.690519 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 23:18:19.690552 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 23:18:19.690627 3637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.687962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:33403 (local address 127.2.76.194:35695)
0809 23:18:19.688207 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:19.688218 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:19.688302 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:19.688439 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:19.688440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:19.688442 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:19.688522 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.688523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.689127 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.689128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.689892 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.689894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.690381 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.690382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.690447 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.690450 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:18:19.690513 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 23:18:19.690551 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":30,"thread_start_us":131,"threads_started":1}
I20260809 23:18:19.690711 3357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.687953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:45057 (local address 127.2.76.194:34075)
0809 23:18:19.688035 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:19.688049 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:19.688148 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:19.688389 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:19.688390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:19.688392 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:19.688477 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.688479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.689148 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.689149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.689902 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:19.689904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.690444 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.690445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:19.690515 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.690518 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:18:19.690588 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 23:18:19.690628 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":63}
I20260809 23:18:19.690709 3619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.688198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:34075 (local address 127.2.76.195:45057)
0809 23:18:19.688261 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:19.688263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:19.688271 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:18:19.688297 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:19.688298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:19.688313 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:19.688385 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:19.688554 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.689058 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.689993 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.690353 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.690445 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.690453 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:19.690597 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 23:18:19.690633 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 23:18:19.690974 3031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271257248fec40efaa31f17b24cb99a6" candidate_uuid: "f1c3e63b1e5445b89c711a532a7fcc74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01654839d0134cce807fa1e2e437b639" is_pre_election: true
I20260809 23:18:19.691133 3031 raft_consensus.cc:2454] T 271257248fec40efaa31f17b24cb99a6 P 01654839d0134cce807fa1e2e437b639 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1c3e63b1e5445b89c711a532a7fcc74 in term 0.
W20260809 23:18:19.691179 3594 tablet.cc:2369] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:18:19.691412 3163 leader_election.cc:302] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01654839d0134cce807fa1e2e437b639, f1c3e63b1e5445b89c711a532a7fcc74; no voters:
I20260809 23:18:19.691166 3514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271257248fec40efaa31f17b24cb99a6" candidate_uuid: "f1c3e63b1e5445b89c711a532a7fcc74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e942593c7d43949b740bc98d5a0758" is_pre_election: true
I20260809 23:18:19.691545 3514 raft_consensus.cc:2454] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1c3e63b1e5445b89c711a532a7fcc74 in term 0.
I20260809 23:18:19.691561 3634 raft_consensus.cc:2781] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:18:19.691612 3634 raft_consensus.cc:482] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:18:19.691651 3634 raft_consensus.cc:3037] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:19.692557 3634 raft_consensus.cc:504] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c3e63b1e5445b89c711a532a7fcc74" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36569 } } peers { permanent_uuid: "85e942593c7d43949b740bc98d5a0758" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45057 } } peers { permanent_uuid: "01654839d0134cce807fa1e2e437b639" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33403 } }
I20260809 23:18:19.692713 3634 leader_election.cc:288] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [CANDIDATE]: Term 1 election: Requested vote from peers 85e942593c7d43949b740bc98d5a0758 (127.2.76.195:45057), 01654839d0134cce807fa1e2e437b639 (127.2.76.193:33403)
I20260809 23:18:19.692832 3514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271257248fec40efaa31f17b24cb99a6" candidate_uuid: "f1c3e63b1e5445b89c711a532a7fcc74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e942593c7d43949b740bc98d5a0758"
I20260809 23:18:19.692916 3514 raft_consensus.cc:3037] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:19.692910 3031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271257248fec40efaa31f17b24cb99a6" candidate_uuid: "f1c3e63b1e5445b89c711a532a7fcc74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01654839d0134cce807fa1e2e437b639"
I20260809 23:18:19.693012 3031 raft_consensus.cc:3037] T 271257248fec40efaa31f17b24cb99a6 P 01654839d0134cce807fa1e2e437b639 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:19.693671 3514 raft_consensus.cc:2454] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1c3e63b1e5445b89c711a532a7fcc74 in term 1.
I20260809 23:18:19.693735 3031 raft_consensus.cc:2454] T 271257248fec40efaa31f17b24cb99a6 P 01654839d0134cce807fa1e2e437b639 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1c3e63b1e5445b89c711a532a7fcc74 in term 1.
I20260809 23:18:19.693902 3166 leader_election.cc:302] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85e942593c7d43949b740bc98d5a0758, f1c3e63b1e5445b89c711a532a7fcc74; no voters:
I20260809 23:18:19.694036 3634 raft_consensus.cc:2781] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:19.694195 3634 raft_consensus.cc:686] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [term 1 LEADER]: Becoming Leader. State: Replica: f1c3e63b1e5445b89c711a532a7fcc74, State: Running, Role: LEADER
I20260809 23:18:19.694296 3634 consensus_queue.cc:237] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c3e63b1e5445b89c711a532a7fcc74" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36569 } } peers { permanent_uuid: "85e942593c7d43949b740bc98d5a0758" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45057 } } peers { permanent_uuid: "01654839d0134cce807fa1e2e437b639" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33403 } }
I20260809 23:18:19.695092 2789 catalog_manager.cc:5495] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1c3e63b1e5445b89c711a532a7fcc74 (127.2.76.194). New cstate: current_term: 1 leader_uuid: "f1c3e63b1e5445b89c711a532a7fcc74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1c3e63b1e5445b89c711a532a7fcc74" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 36569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85e942593c7d43949b740bc98d5a0758" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01654839d0134cce807fa1e2e437b639" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33403 } health_report { overall_health: UNKNOWN } } }
I20260809 23:18:19.755123 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.752866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38410 (local address 127.2.76.194:36569)
0809 23:18:19.752928 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:19.752930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:19.752956 (+ 26us) server_negotiation.cc:407] Connection header received
0809 23:18:19.752996 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:19.752997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:19.753015 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:19.753085 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:19.753210 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.753651 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.754398 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.754576 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.754643 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.754806 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:19.754898 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:19.754984 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:19.755001 (+ 17us) server_negotiation.cc:299] Negotiation successful
0809 23:18:19.755021 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 23:18:19.756810 3514 raft_consensus.cc:1261] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758 [term 1 FOLLOWER]: Refusing update from remote peer f1c3e63b1e5445b89c711a532a7fcc74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:19.756812 3031 raft_consensus.cc:1261] T 271257248fec40efaa31f17b24cb99a6 P 01654839d0134cce807fa1e2e437b639 [term 1 FOLLOWER]: Refusing update from remote peer f1c3e63b1e5445b89c711a532a7fcc74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:19.757130 3640 consensus_queue.cc:1035] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01654839d0134cce807fa1e2e437b639" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 33403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:19.757252 3634 consensus_queue.cc:1035] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e942593c7d43949b740bc98d5a0758" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 45057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:19.759699 3643 mvcc.cc:205] Tried to move back new op lower bound from 7316756479001325568 to 7316756478748274688. Current Snapshot: MvccSnapshot[applied={T|T < 7316756479001325568}]
I20260809 23:18:19.760141 3645 mvcc.cc:205] Tried to move back new op lower bound from 7316756479001325568 to 7316756478748274688. Current Snapshot: MvccSnapshot[applied={T|T < 7316756479001325568 or (T in {7316756479001325568})}]
I20260809 23:18:19.760872 3644 mvcc.cc:205] Tried to move back new op lower bound from 7316756479001325568 to 7316756478748274688. Current Snapshot: MvccSnapshot[applied={T|T < 7316756479001325568 or (T in {7316756479001325568})}]
I20260809 23:18:19.761718 2789 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:18:19.761807 2789 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:18:19.762519 2789 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 552a257260184990b84c88c20b48bf83: Sending DeleteTablet for 3 replicas of tablet 271257248fec40efaa31f17b24cb99a6
I20260809 23:18:19.762830 2993 tablet_service.cc:1514] Processing DeleteTablet for tablet 271257248fec40efaa31f17b24cb99a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:19 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:56168
I20260809 23:18:19.762837 3245 tablet_service.cc:1514] Processing DeleteTablet for tablet 271257248fec40efaa31f17b24cb99a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:19 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:38398
I20260809 23:18:19.763038 3484 tablet_service.cc:1514] Processing DeleteTablet for tablet 271257248fec40efaa31f17b24cb99a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:19 UTC) from {username='kudu', principal='kudu/127.2.76.254@KRBTEST.COM'} at 127.0.0.1:35140
I20260809 23:18:19.763248 3658 tablet_replica.cc:330] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758: stopping tablet replica
I20260809 23:18:19.763389 3658 raft_consensus.cc:2229] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:18:19.763430 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 2890
I20260809 23:18:19.764067 3658 raft_consensus.cc:2258] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:18:19.767063 3658 ts_tablet_manager.cc:1905] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:18:19.768855 3658 ts_tablet_manager.cc:1918] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:18:19.768918 3658 log.cc:1196] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/271257248fec40efaa31f17b24cb99a6
I20260809 23:18:19.769130 3658 ts_tablet_manager.cc:1939] T 271257248fec40efaa31f17b24cb99a6 P 85e942593c7d43949b740bc98d5a0758: Deleting consensus metadata
I20260809 23:18:19.770140 2772 catalog_manager.cc:4841] TS 85e942593c7d43949b740bc98d5a0758 (127.2.76.195:45057): tablet 271257248fec40efaa31f17b24cb99a6 (table test-table [id=c5646ca889dd4c03a942ce786763b121]) successfully deleted
I20260809 23:18:19.775664 3657 tablet_replica.cc:330] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74: stopping tablet replica
I20260809 23:18:19.775768 3657 raft_consensus.cc:2229] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:18:19.775854 3657 raft_consensus.cc:2258] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:18:19.776280 3657 ts_tablet_manager.cc:1905] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:18:19.778342 3657 ts_tablet_manager.cc:1918] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:18:19.778424 3657 log.cc:1196] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/271257248fec40efaa31f17b24cb99a6
I20260809 23:18:19.778693 3657 ts_tablet_manager.cc:1939] T 271257248fec40efaa31f17b24cb99a6 P f1c3e63b1e5445b89c711a532a7fcc74: Deleting consensus metadata
I20260809 23:18:19.779075 2771 catalog_manager.cc:4841] TS f1c3e63b1e5445b89c711a532a7fcc74 (127.2.76.194:36569): tablet 271257248fec40efaa31f17b24cb99a6 (table test-table [id=c5646ca889dd4c03a942ce786763b121]) successfully deleted
W20260809 23:18:19.782696 2769 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:33403 (error 108)
I20260809 23:18:19.782955 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 3122
I20260809 23:18:19.783142 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.783033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:33403 (local address 127.0.0.1:56182)
0809 23:18:19.783074 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:19.783108 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:33403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 23:18:19.783231 2769 catalog_manager.cc:4584] TS 01654839d0134cce807fa1e2e437b639 (127.2.76.193:33403): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 271257248fec40efaa31f17b24cb99a6: Network error: Client connection negotiation failed: client connection to 127.2.76.193:33403: connect: Connection refused (error 111)
I20260809 23:18:19.790828 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 3362
I20260809 23:18:19.800879 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 2739
2026-08-09T23:18:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2141 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T23:18:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:19Z Disabled control of system clock
I20260809 23:18:19.830251 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:35115
--webserver_interface=127.2.76.254
--webserver_port=0
--builtin_ntp_servers=127.2.76.212:43985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:35115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:18:19.903484 3664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:19.903806 3664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:19.903837 3664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:19.905073 3664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:19.905114 3664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:19.905126 3664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:19.905138 3664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:19.905167 3664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:19.906327 3664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:35115
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:35115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:19.906551 3664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:19.906810 3664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:19.910975 3669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:19.910995 3671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:19.911020 3679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:19.911150 3664 server_base.cc:1034] running on GCE node
I20260809 23:18:19.911388 3664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:19.911609 3664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:19.912734 3664 hybrid_clock.cc:648] HybridClock initialized: now 1786317499912718 us; error 31 us; skew 500 ppm
I20260809 23:18:19.913853 3664 webserver.cc:460] Webserver started at http://127.2.76.254:39749/ using document root <none> and password file <none>
I20260809 23:18:19.914068 3664 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:19.914110 3664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:19.914238 3664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:19.915133 3664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "4e42596c210541179ef51014ec88ce42"
format_stamp: "Formatted at 2026-08-09 23:18:19 on dist-test-slave-7vjv"
server_key: "5b80dbe29f5b8eda224586418c403e77"
server_key_iv: "492144be18cdffd86873951645fc226c"
server_key_version: "encryptionkey@0"
I20260809 23:18:19.915449 3664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "4e42596c210541179ef51014ec88ce42"
format_stamp: "Formatted at 2026-08-09 23:18:19 on dist-test-slave-7vjv"
server_key: "5b80dbe29f5b8eda224586418c403e77"
server_key_iv: "492144be18cdffd86873951645fc226c"
server_key_version: "encryptionkey@0"
I20260809 23:18:19.916657 3664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:18:19.917253 3698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:19.917382 3664 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:19.917454 3664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "4e42596c210541179ef51014ec88ce42"
format_stamp: "Formatted at 2026-08-09 23:18:19 on dist-test-slave-7vjv"
server_key: "5b80dbe29f5b8eda224586418c403e77"
server_key_iv: "492144be18cdffd86873951645fc226c"
server_key_version: "encryptionkey@0"
I20260809 23:18:19.917505 3664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:19.925976 3664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:19.926179 3664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:19.926267 3664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:19.929806 3664 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:35115
I20260809 23:18:19.929874 3796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:35115 every 8 connection(s)
I20260809 23:18:19.930140 3664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:19.930922 3797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:19.932803 3797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42: Bootstrap starting.
I20260809 23:18:19.933272 3797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:19.933501 3797 log.cc:824] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:19.933990 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 3664
I20260809 23:18:19.934058 3797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42: No bootstrap required, opened a new log
I20260809 23:18:19.934085 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:19.934227 2355 external_mini_cluster.cc:1428] Setting key 71aaf1c8b571a4f0086fac6ba66a145d
I20260809 23:18:19.936672 3797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e42596c210541179ef51014ec88ce42" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35115 } }
I20260809 23:18:19.936817 3797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:19.936844 3797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e42596c210541179ef51014ec88ce42, State: Initialized, Role: FOLLOWER
I20260809 23:18:19.936952 3797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e42596c210541179ef51014ec88ce42" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35115 } }
I20260809 23:18:19.937049 3797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:19.937093 3797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:19.937137 3797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:19.937597 3797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e42596c210541179ef51014ec88ce42" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35115 } }
I20260809 23:18:19.937697 3797 leader_election.cc:302] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e42596c210541179ef51014ec88ce42; no voters:
I20260809 23:18:19.937845 3797 leader_election.cc:288] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:19.937904 3812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:19.938037 3812 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [term 1 LEADER]: Becoming Leader. State: Replica: 4e42596c210541179ef51014ec88ce42, State: Running, Role: LEADER
I20260809 23:18:19.938066 3797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:19.938122 3812 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e42596c210541179ef51014ec88ce42" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35115 } }
I20260809 23:18:19.938395 3812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e42596c210541179ef51014ec88ce42. Latest consensus state: current_term: 1 leader_uuid: "4e42596c210541179ef51014ec88ce42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e42596c210541179ef51014ec88ce42" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35115 } } }
I20260809 23:18:19.938463 3812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:19.938695 3813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e42596c210541179ef51014ec88ce42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e42596c210541179ef51014ec88ce42" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 35115 } } }
I20260809 23:18:19.938797 3813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:19.938992 3821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:19.939455 3821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:19.939803 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:19.934656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59432 (local address 127.2.76.254:35115)
0809 23:18:19.934851 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:19.934858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:19.934887 (+ 29us) server_negotiation.cc:407] Connection header received
0809 23:18:19.935302 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:19.935310 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:19.935376 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:19.935521 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:19.935640 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.936944 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.937714 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:19.938267 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:19.938378 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:19.939025 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:19.939028 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:19.939028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:19.939220 (+ 192us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:19.939255 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:19.939269 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:19.939323 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:19.939509 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 23:18:19.939600 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260809 23:18:19.940865 3821 catalog_manager.cc:1347] Generated new cluster ID: 35819aa1bae048418e05b27a5c9f2bc1
I20260809 23:18:19.940918 3821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:19.948096 3821 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:19.948544 3821 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:19.957643 3821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42: Generated new TSK 0
I20260809 23:18:19.957773 3821 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:19.962447 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.193:0
--local_ip_for_outbound_sockets=127.2.76.193
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35115
--builtin_ntp_servers=127.2.76.212:43985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:18:20.050421 3836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:20.050645 3836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:20.050681 3836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:20.051769 3836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:20.051824 3836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:20.051862 3836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.193
I20260809 23:18:20.053092 3836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.76.193
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.193
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:20.053303 3836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:20.053566 3836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:20.057906 3849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:20.057941 3842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:20.057960 3845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:20.058760 3836 server_base.cc:1034] running on GCE node
I20260809 23:18:20.058960 3836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:20.059186 3836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:20.060329 3836 hybrid_clock.cc:648] HybridClock initialized: now 1786317500060309 us; error 36 us; skew 500 ppm
I20260809 23:18:20.061512 3836 webserver.cc:460] Webserver started at http://127.2.76.193:40049/ using document root <none> and password file <none>
I20260809 23:18:20.061731 3836 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:20.061772 3836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:20.061893 3836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:20.062913 3836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/data/instance:
uuid: "05b1551773d14c5c91db0176b81c0d5e"
format_stamp: "Formatted at 2026-08-09 23:18:20 on dist-test-slave-7vjv"
server_key: "071832c5e9088149430bbc1a6b345912"
server_key_iv: "09e46c61a3480bc5938883be43ece887"
server_key_version: "encryptionkey@0"
I20260809 23:18:20.063272 3836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance:
uuid: "05b1551773d14c5c91db0176b81c0d5e"
format_stamp: "Formatted at 2026-08-09 23:18:20 on dist-test-slave-7vjv"
server_key: "071832c5e9088149430bbc1a6b345912"
server_key_iv: "09e46c61a3480bc5938883be43ece887"
server_key_version: "encryptionkey@0"
I20260809 23:18:20.064433 3836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:18:20.065102 3858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:20.065277 3836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:20.065376 3836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/wal
uuid: "05b1551773d14c5c91db0176b81c0d5e"
format_stamp: "Formatted at 2026-08-09 23:18:20 on dist-test-slave-7vjv"
server_key: "071832c5e9088149430bbc1a6b345912"
server_key_iv: "09e46c61a3480bc5938883be43ece887"
server_key_version: "encryptionkey@0"
I20260809 23:18:20.065448 3836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:20.085721 3836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:20.085943 3836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:20.086041 3836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:20.086206 3836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:20.086440 3836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:20.086469 3836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:20.086489 3836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:20.086508 3836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:20.092160 3836 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.193:44525
I20260809 23:18:20.092223 4048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.193:44525 every 8 connection(s)
I20260809 23:18:20.092494 3836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/data/info.pb
I20260809 23:18:20.097002 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.093068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:42409 (local address 127.2.76.254:35115)
0809 23:18:20.093147 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:20.093149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:20.093335 (+ 186us) server_negotiation.cc:407] Connection header received
0809 23:18:20.093762 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:20.093762 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:20.093775 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:20.093844 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:20.094405 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.094873 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.095880 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.096057 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.096196 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.096813 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:20.096814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:20.096814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:20.096818 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:20.096830 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:20.096834 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:20.096870 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:20.096931 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 23:18:20.096949 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 23:18:20.097196 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.093038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:35115 (local address 127.2.76.193:42409)
0809 23:18:20.093209 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:20.093236 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:20.093682 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:20.093888 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:20.093891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:20.093897 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:20.094336 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.094339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.094951 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.094952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.095712 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.095713 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.096212 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.096212 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.096311 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.096702 (+ 391us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:18:20.096702 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:20.096725 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:18:20.096743 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:20.096876 (+ 133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:20.096878 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:20.096880 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:18:20.096919 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 23:18:20.096983 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260809 23:18:20.097678 4049 heartbeater.cc:347] Connected to a master server at 127.2.76.254:35115
I20260809 23:18:20.097766 4049 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:20.097919 4049 heartbeater.cc:511] Master 127.2.76.254:35115 requested a full tablet report, sending...
I20260809 23:18:20.098394 3727 ts_manager.cc:194] Registered new tserver with Master: 05b1551773d14c5c91db0176b81c0d5e (127.2.76.193:44525)
I20260809 23:18:20.098471 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 3836
I20260809 23:18:20.098577 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-0/wal/instance
I20260809 23:18:20.098773 2355 external_mini_cluster.cc:1428] Setting key 2d3218efc322ab6369219630411e7338
I20260809 23:18:20.099258 3727 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.76.193:42409
I20260809 23:18:20.099845 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.194:0
--local_ip_for_outbound_sockets=127.2.76.194
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35115
--builtin_ntp_servers=127.2.76.212:43985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:18:20.171567 4056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:20.171734 4056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:20.171753 4056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:20.172748 4056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:20.172778 4056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:20.172801 4056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.194
I20260809 23:18:20.173914 4056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.76.194
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.194
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:20.174064 4056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:20.174304 4056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:20.178503 4065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:20.178491 4061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:20.178493 4062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:20.178750 4056 server_base.cc:1034] running on GCE node
I20260809 23:18:20.178954 4056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:20.179193 4056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:20.180351 4056 hybrid_clock.cc:648] HybridClock initialized: now 1786317500180332 us; error 28 us; skew 500 ppm
I20260809 23:18:20.181499 4056 webserver.cc:460] Webserver started at http://127.2.76.194:42019/ using document root <none> and password file <none>
I20260809 23:18:20.181715 4056 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:20.181775 4056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:20.181895 4056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:20.182806 4056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/data/instance:
uuid: "586c1e0d9f354de9a611fa19a0955b72"
format_stamp: "Formatted at 2026-08-09 23:18:20 on dist-test-slave-7vjv"
server_key: "d9647de2d03f2e49322f527f3955a464"
server_key_iv: "be3a404aa79e358e83a3db8ec4d668de"
server_key_version: "encryptionkey@0"
I20260809 23:18:20.183152 4056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance:
uuid: "586c1e0d9f354de9a611fa19a0955b72"
format_stamp: "Formatted at 2026-08-09 23:18:20 on dist-test-slave-7vjv"
server_key: "d9647de2d03f2e49322f527f3955a464"
server_key_iv: "be3a404aa79e358e83a3db8ec4d668de"
server_key_version: "encryptionkey@0"
I20260809 23:18:20.184451 4056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:18:20.185086 4077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:20.185256 4056 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:20.185343 4056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/wal
uuid: "586c1e0d9f354de9a611fa19a0955b72"
format_stamp: "Formatted at 2026-08-09 23:18:20 on dist-test-slave-7vjv"
server_key: "d9647de2d03f2e49322f527f3955a464"
server_key_iv: "be3a404aa79e358e83a3db8ec4d668de"
server_key_version: "encryptionkey@0"
I20260809 23:18:20.185415 4056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:20.198022 4056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:20.198240 4056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:20.198359 4056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:20.198537 4056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:20.198800 4056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:20.198845 4056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:20.198879 4056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:20.198902 4056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:20.204008 4056 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.194:34605
I20260809 23:18:20.204069 4272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.194:34605 every 8 connection(s)
I20260809 23:18:20.204344 4056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/data/info.pb
I20260809 23:18:20.204771 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 4056
I20260809 23:18:20.204875 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/wal/instance
I20260809 23:18:20.205102 2355 external_mini_cluster.cc:1428] Setting key f34e57c8fa15046318057855137f8e4e
I20260809 23:18:20.206009 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.76.195:0
--local_ip_for_outbound_sockets=127.2.76.195
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35115
--builtin_ntp_servers=127.2.76.212:43985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 23:18:20.209827 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.205012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.194:34579 (local address 127.2.76.254:35115)
0809 23:18:20.205089 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:20.205091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:20.205312 (+ 221us) server_negotiation.cc:407] Connection header received
0809 23:18:20.205871 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:20.205872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:20.205879 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:20.205915 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:20.206659 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.207066 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.208481 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.208759 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.208830 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.209563 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:20.209564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:20.209565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:20.209569 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:20.209586 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:20.209590 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:20.209637 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:20.209698 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 23:18:20.209726 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 23:18:20.209895 4280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.204979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:35115 (local address 127.2.76.194:34579)
0809 23:18:20.205170 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:20.205208 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:20.205781 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:20.205990 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:20.206001 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:20.206010 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:20.206577 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.206583 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.207145 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.207146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.208376 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.208380 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.208903 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.208904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.209024 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.209446 (+ 422us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:18:20.209446 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:20.209467 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:18:20.209485 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:20.209651 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:20.209655 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:20.209657 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:18:20.209696 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 23:18:20.209760 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260809 23:18:20.210281 4273 heartbeater.cc:347] Connected to a master server at 127.2.76.254:35115
I20260809 23:18:20.210373 4273 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:20.210520 4273 heartbeater.cc:511] Master 127.2.76.254:35115 requested a full tablet report, sending...
I20260809 23:18:20.210884 3727 ts_manager.cc:194] Registered new tserver with Master: 586c1e0d9f354de9a611fa19a0955b72 (127.2.76.194:34605)
I20260809 23:18:20.211208 3727 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.76.194:34579
W20260809 23:18:20.282322 4282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:20.282490 4282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:20.282510 4282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:20.283560 4282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:20.283604 4282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:20.283631 4282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.76.195
I20260809 23:18:20.285018 4282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.76.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.76.195
--webserver_port=0
--tserver_master_addrs=127.2.76.254:35115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.76.195
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:20.285250 4282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:20.285533 4282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:20.289934 4292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:20.289978 4291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:20.290092 4282 server_base.cc:1034] running on GCE node
W20260809 23:18:20.289934 4294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:20.290391 4282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:20.290594 4282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:20.291742 4282 hybrid_clock.cc:648] HybridClock initialized: now 1786317500291727 us; error 46 us; skew 500 ppm
I20260809 23:18:20.292882 4282 webserver.cc:460] Webserver started at http://127.2.76.195:40591/ using document root <none> and password file <none>
I20260809 23:18:20.293076 4282 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:20.293114 4282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:20.293200 4282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:20.294071 4282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/data/instance:
uuid: "bf6a5661b3cb457ea03815e04948c29b"
format_stamp: "Formatted at 2026-08-09 23:18:20 on dist-test-slave-7vjv"
server_key: "27906969e24d79e7a3c5b9643d63223a"
server_key_iv: "f52402818a005c59bd3c4a1a1472907b"
server_key_version: "encryptionkey@0"
I20260809 23:18:20.294363 4282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance:
uuid: "bf6a5661b3cb457ea03815e04948c29b"
format_stamp: "Formatted at 2026-08-09 23:18:20 on dist-test-slave-7vjv"
server_key: "27906969e24d79e7a3c5b9643d63223a"
server_key_iv: "f52402818a005c59bd3c4a1a1472907b"
server_key_version: "encryptionkey@0"
I20260809 23:18:20.295634 4282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:18:20.296334 4303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:20.296464 4282 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:18:20.296538 4282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/wal
uuid: "bf6a5661b3cb457ea03815e04948c29b"
format_stamp: "Formatted at 2026-08-09 23:18:20 on dist-test-slave-7vjv"
server_key: "27906969e24d79e7a3c5b9643d63223a"
server_key_iv: "f52402818a005c59bd3c4a1a1472907b"
server_key_version: "encryptionkey@0"
I20260809 23:18:20.296584 4282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:20.305845 4282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:20.306020 4282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:20.306100 4282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:20.306238 4282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:18:20.306435 4282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:18:20.306461 4282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:20.306478 4282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:18:20.306488 4282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:20.311405 4282 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.195:44669
I20260809 23:18:20.311473 4502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.195:44669 every 8 connection(s)
I20260809 23:18:20.311776 4282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/data/info.pb
I20260809 23:18:20.316308 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.312421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.195:48193 (local address 127.2.76.254:35115)
0809 23:18:20.312462 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:20.312464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:20.312670 (+ 206us) server_negotiation.cc:407] Connection header received
0809 23:18:20.313076 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:20.313077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:20.313090 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:20.313133 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:20.313675 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.314213 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.315103 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.315288 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.315352 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.316009 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:20.316011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:20.316011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:20.316016 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:20.316029 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:20.316033 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:20.316083 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:20.316196 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 23:18:20.316237 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:18:20.316478 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.312401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:35115 (local address 127.2.76.195:48193)
0809 23:18:20.312582 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:20.312607 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:20.312975 (+ 368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:20.313185 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:20.313188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:20.313196 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:20.313590 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.313593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.314295 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.314296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.315014 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.315016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.315360 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.315362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.315479 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.315878 (+ 399us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:18:20.315879 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:20.315902 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:18:20.315920 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:20.316086 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:20.316106 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:20.316109 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:18:20.316185 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 23:18:20.316290 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":94,"threads_started":1}
I20260809 23:18:20.316957 4504 heartbeater.cc:347] Connected to a master server at 127.2.76.254:35115
I20260809 23:18:20.317039 4504 heartbeater.cc:464] Registering TS with master...
I20260809 23:18:20.317157 4504 heartbeater.cc:511] Master 127.2.76.254:35115 requested a full tablet report, sending...
I20260809 23:18:20.317466 3727 ts_manager.cc:194] Registered new tserver with Master: bf6a5661b3cb457ea03815e04948c29b (127.2.76.195:44669)
I20260809 23:18:20.317803 3727 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.76.195:48193
I20260809 23:18:20.321652 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 4282
I20260809 23:18:20.321748 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/wal/instance
I20260809 23:18:20.321933 2355 external_mini_cluster.cc:1428] Setting key 0dba4343c86753cd89ef934e17490810
I20260809 23:18:20.322615 2355 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:18:20.326256 3805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.323793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59440 (local address 127.2.76.254:35115)
0809 23:18:20.323824 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:20.323826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:20.323976 (+ 150us) server_negotiation.cc:407] Connection header received
0809 23:18:20.324062 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:20.324062 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:20.324072 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:20.324178 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:20.324326 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.324741 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.325392 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.325551 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.325609 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.326060 (+ 451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:20.326061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:20.326062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:20.326065 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:20.326074 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:20.326078 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:20.326123 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:20.326173 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 23:18:20.326189 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 23:18:20.327376 3727 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:18:20.328924 3727 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:18:20.334596 4518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.330723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:44525 (local address 127.0.0.1:40758)
0809 23:18:20.330926 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:20.330966 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:20.331028 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:20.331385 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:20.331388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:20.331390 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:20.331455 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.331457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.332920 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.332921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.333655 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.333658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.334323 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.334324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.334394 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.334405 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 23:18:20.334464 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 23:18:20.334526 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":56,"threads_started":1}
I20260809 23:18:20.334771 4524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.331137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49524 (local address 127.2.76.194:34605)
0809 23:18:20.331323 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:20.331327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:20.331337 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:18:20.331378 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:20.331380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:20.331396 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:20.331469 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:20.331588 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.332456 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.333728 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.334379 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.334445 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.334519 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:20.334551 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 23:18:20.334585 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260809 23:18:20.334906 4519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.330761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:44669 (local address 127.0.0.1:39868)
0809 23:18:20.331124 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:20.331132 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:20.331169 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:20.331528 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:20.331528 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:20.331530 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:20.331563 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.331564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.332896 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.332897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.333868 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.333871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.334652 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.334653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.334712 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.334716 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:18:20.334744 (+ 28us) client_negotiation.cc:240] Negotiation successful
0809 23:18:20.334784 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":2,"thread_start_us":99,"threads_started":1}
I20260809 23:18:20.334811 4517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.330760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:34605 (local address 127.0.0.1:49524)
0809 23:18:20.331120 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:20.331129 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:20.331170 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:20.331477 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:20.331477 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:20.331480 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:20.331533 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.331535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.332555 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.332556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.333638 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.333640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.334450 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.334451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.334499 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.334501 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:18:20.334538 (+ 37us) client_negotiation.cc:240] Negotiation successful
0809 23:18:20.334566 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":98,"threads_started":1}
I20260809 23:18:20.335070 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.330981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.2.76.195:44669)
0809 23:18:20.331363 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:20.331366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:20.331375 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:18:20.331403 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:20.331404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:20.331417 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:20.331667 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:20.331710 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.332796 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.333949 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.334565 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.334853 (+ 288us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.334937 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:20.334962 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 23:18:20.335004 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":78,"threads_started":1}
I20260809 23:18:20.335428 4521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.330902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.2.76.193:44525)
0809 23:18:20.331122 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:20.331149 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:20.331162 (+ 13us) server_negotiation.cc:407] Connection header received
0809 23:18:20.331206 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:20.331209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:20.331240 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:20.331563 (+ 323us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:20.331653 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.332822 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.333743 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.334239 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.334320 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.334401 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:20.335337 (+ 936us) server_negotiation.cc:299] Negotiation successful
0809 23:18:20.335372 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":112,"threads_started":1}
I20260809 23:18:20.335717 4380 tablet_service.cc:1467] Processing CreateTablet for tablet 3baa40fc86804f73a219d748b99cb824 (DEFAULT_TABLE table=test-table [id=9389a038e6694e42bc421045c6f651e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:20.336028 4380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3baa40fc86804f73a219d748b99cb824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:20.336081 3942 tablet_service.cc:1467] Processing CreateTablet for tablet 3baa40fc86804f73a219d748b99cb824 (DEFAULT_TABLE table=test-table [id=9389a038e6694e42bc421045c6f651e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:20.336439 3942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3baa40fc86804f73a219d748b99cb824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:20.337020 4160 tablet_service.cc:1467] Processing CreateTablet for tablet 3baa40fc86804f73a219d748b99cb824 (DEFAULT_TABLE table=test-table [id=9389a038e6694e42bc421045c6f651e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:18:20.337495 4160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3baa40fc86804f73a219d748b99cb824. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:20.339960 4529 tablet_bootstrap.cc:492] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e: Bootstrap starting.
I20260809 23:18:20.340608 4528 tablet_bootstrap.cc:492] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b: Bootstrap starting.
I20260809 23:18:20.340675 4529 tablet_bootstrap.cc:654] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:20.340993 4529 log.cc:824] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:20.341471 4528 tablet_bootstrap.cc:654] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:20.341676 4528 log.cc:824] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:20.341881 4533 tablet_bootstrap.cc:492] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72: Bootstrap starting.
I20260809 23:18:20.342463 4528 tablet_bootstrap.cc:492] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b: No bootstrap required, opened a new log
I20260809 23:18:20.342491 4533 tablet_bootstrap.cc:654] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:20.342533 4528 ts_tablet_manager.cc:1397] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:18:20.342819 4533 log.cc:824] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:20.343118 4529 tablet_bootstrap.cc:492] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e: No bootstrap required, opened a new log
I20260809 23:18:20.343202 4529 ts_tablet_manager.cc:1397] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 23:18:20.343719 4533 tablet_bootstrap.cc:492] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72: No bootstrap required, opened a new log
I20260809 23:18:20.343787 4533 ts_tablet_manager.cc:1397] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:18:20.344969 4529 raft_consensus.cc:348] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1551773d14c5c91db0176b81c0d5e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44525 } } peers { permanent_uuid: "586c1e0d9f354de9a611fa19a0955b72" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34605 } } peers { permanent_uuid: "bf6a5661b3cb457ea03815e04948c29b" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 44669 } }
I20260809 23:18:20.345116 4529 raft_consensus.cc:374] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:20.345158 4529 raft_consensus.cc:729] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b1551773d14c5c91db0176b81c0d5e, State: Initialized, Role: FOLLOWER
I20260809 23:18:20.345093 4528 raft_consensus.cc:348] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1551773d14c5c91db0176b81c0d5e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44525 } } peers { permanent_uuid: "586c1e0d9f354de9a611fa19a0955b72" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34605 } } peers { permanent_uuid: "bf6a5661b3cb457ea03815e04948c29b" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 44669 } }
I20260809 23:18:20.345238 4528 raft_consensus.cc:374] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:20.345281 4528 raft_consensus.cc:729] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf6a5661b3cb457ea03815e04948c29b, State: Initialized, Role: FOLLOWER
I20260809 23:18:20.345258 4529 consensus_queue.cc:260] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1551773d14c5c91db0176b81c0d5e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44525 } } peers { permanent_uuid: "586c1e0d9f354de9a611fa19a0955b72" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34605 } } peers { permanent_uuid: "bf6a5661b3cb457ea03815e04948c29b" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 44669 } }
I20260809 23:18:20.345399 4528 consensus_queue.cc:260] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1551773d14c5c91db0176b81c0d5e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44525 } } peers { permanent_uuid: "586c1e0d9f354de9a611fa19a0955b72" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34605 } } peers { permanent_uuid: "bf6a5661b3cb457ea03815e04948c29b" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 44669 } }
I20260809 23:18:20.345536 4529 ts_tablet_manager.cc:1428] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260809 23:18:20.345613 4049 heartbeater.cc:503] Master 127.2.76.254:35115 was elected leader, sending a full tablet report...
I20260809 23:18:20.345925 4528 ts_tablet_manager.cc:1428] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:18:20.346071 4504 heartbeater.cc:503] Master 127.2.76.254:35115 was elected leader, sending a full tablet report...
I20260809 23:18:20.346201 4533 raft_consensus.cc:348] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1551773d14c5c91db0176b81c0d5e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44525 } } peers { permanent_uuid: "586c1e0d9f354de9a611fa19a0955b72" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34605 } } peers { permanent_uuid: "bf6a5661b3cb457ea03815e04948c29b" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 44669 } }
I20260809 23:18:20.346365 4533 raft_consensus.cc:374] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:20.346412 4533 raft_consensus.cc:729] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 586c1e0d9f354de9a611fa19a0955b72, State: Initialized, Role: FOLLOWER
I20260809 23:18:20.346539 4533 consensus_queue.cc:260] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1551773d14c5c91db0176b81c0d5e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44525 } } peers { permanent_uuid: "586c1e0d9f354de9a611fa19a0955b72" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34605 } } peers { permanent_uuid: "bf6a5661b3cb457ea03815e04948c29b" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 44669 } }
I20260809 23:18:20.346820 4533 ts_tablet_manager.cc:1428] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:18:20.346937 4273 heartbeater.cc:503] Master 127.2.76.254:35115 was elected leader, sending a full tablet report...
I20260809 23:18:20.368867 4537 raft_consensus.cc:482] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:18:20.368988 4537 raft_consensus.cc:504] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1551773d14c5c91db0176b81c0d5e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44525 } } peers { permanent_uuid: "586c1e0d9f354de9a611fa19a0955b72" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34605 } } peers { permanent_uuid: "bf6a5661b3cb457ea03815e04948c29b" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 44669 } }
I20260809 23:18:20.369295 4537 leader_election.cc:288] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 586c1e0d9f354de9a611fa19a0955b72 (127.2.76.194:34605), bf6a5661b3cb457ea03815e04948c29b (127.2.76.195:44669)
I20260809 23:18:20.371871 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.369454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.195:44669 (local address 127.2.76.193:51753)
0809 23:18:20.369627 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:20.369637 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:20.369692 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:20.369836 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:20.369837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:20.369838 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:20.369933 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.369934 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.370455 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.370456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.371145 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.371147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.371619 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.371620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.371711 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.371713 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:18:20.371752 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 23:18:20.371793 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":12,"thread_start_us":111,"threads_started":1}
I20260809 23:18:20.371876 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.369501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:51753 (local address 127.2.76.195:44669)
0809 23:18:20.369547 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:20.369549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:20.369695 (+ 146us) server_negotiation.cc:407] Connection header received
0809 23:18:20.369749 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:20.369749 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:20.369759 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:20.369813 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:20.369983 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.370379 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.371222 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.371544 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.371592 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.371597 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:20.371775 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 23:18:20.371817 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 23:18:20.371913 4524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.369496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.76.193:59073 (local address 127.2.76.194:34605)
0809 23:18:20.369534 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:20.369536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:20.369545 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:18:20.369599 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:20.369600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:20.369609 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:20.369669 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:18:20.369838 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.370383 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.371272 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.371559 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.371634 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.371657 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:20.371815 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 23:18:20.371850 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:18:20.371960 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.369443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.194:34605 (local address 127.2.76.193:59073)
0809 23:18:20.369497 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:20.369507 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:20.369546 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:20.369705 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:20.369706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:20.369708 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:18:20.369777 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.369779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.370474 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.370475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.371197 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:20.371199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.371679 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.371680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:20.371733 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.371736 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:18:20.371800 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 23:18:20.371843 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 23:18:20.372421 4422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3baa40fc86804f73a219d748b99cb824" candidate_uuid: "05b1551773d14c5c91db0176b81c0d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf6a5661b3cb457ea03815e04948c29b" is_pre_election: true
I20260809 23:18:20.372447 4194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3baa40fc86804f73a219d748b99cb824" candidate_uuid: "05b1551773d14c5c91db0176b81c0d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "586c1e0d9f354de9a611fa19a0955b72" is_pre_election: true
I20260809 23:18:20.372552 4422 raft_consensus.cc:2454] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b1551773d14c5c91db0176b81c0d5e in term 0.
I20260809 23:18:20.372567 4194 raft_consensus.cc:2454] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05b1551773d14c5c91db0176b81c0d5e in term 0.
I20260809 23:18:20.372750 3860 leader_election.cc:302] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b1551773d14c5c91db0176b81c0d5e, bf6a5661b3cb457ea03815e04948c29b; no voters:
I20260809 23:18:20.372874 4537 raft_consensus.cc:2781] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:18:20.372928 4537 raft_consensus.cc:482] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:18:20.372959 4537 raft_consensus.cc:3037] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:20.373572 4537 raft_consensus.cc:504] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1551773d14c5c91db0176b81c0d5e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44525 } } peers { permanent_uuid: "586c1e0d9f354de9a611fa19a0955b72" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34605 } } peers { permanent_uuid: "bf6a5661b3cb457ea03815e04948c29b" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 44669 } }
I20260809 23:18:20.373716 4537 leader_election.cc:288] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [CANDIDATE]: Term 1 election: Requested vote from peers 586c1e0d9f354de9a611fa19a0955b72 (127.2.76.194:34605), bf6a5661b3cb457ea03815e04948c29b (127.2.76.195:44669)
I20260809 23:18:20.373855 4422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3baa40fc86804f73a219d748b99cb824" candidate_uuid: "05b1551773d14c5c91db0176b81c0d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf6a5661b3cb457ea03815e04948c29b"
I20260809 23:18:20.373862 4194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3baa40fc86804f73a219d748b99cb824" candidate_uuid: "05b1551773d14c5c91db0176b81c0d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "586c1e0d9f354de9a611fa19a0955b72"
I20260809 23:18:20.373916 4422 raft_consensus.cc:3037] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:20.373932 4194 raft_consensus.cc:3037] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:20.374508 4422 raft_consensus.cc:2454] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b1551773d14c5c91db0176b81c0d5e in term 1.
I20260809 23:18:20.374595 4194 raft_consensus.cc:2454] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05b1551773d14c5c91db0176b81c0d5e in term 1.
I20260809 23:18:20.374652 3860 leader_election.cc:302] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b1551773d14c5c91db0176b81c0d5e, bf6a5661b3cb457ea03815e04948c29b; no voters:
I20260809 23:18:20.374737 4537 raft_consensus.cc:2781] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:20.374891 4537 raft_consensus.cc:686] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [term 1 LEADER]: Becoming Leader. State: Replica: 05b1551773d14c5c91db0176b81c0d5e, State: Running, Role: LEADER
I20260809 23:18:20.374979 4537 consensus_queue.cc:237] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1551773d14c5c91db0176b81c0d5e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44525 } } peers { permanent_uuid: "586c1e0d9f354de9a611fa19a0955b72" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34605 } } peers { permanent_uuid: "bf6a5661b3cb457ea03815e04948c29b" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 44669 } }
I20260809 23:18:20.375617 3726 catalog_manager.cc:5495] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 05b1551773d14c5c91db0176b81c0d5e (127.2.76.193). New cstate: current_term: 1 leader_uuid: "05b1551773d14c5c91db0176b81c0d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1551773d14c5c91db0176b81c0d5e" member_type: VOTER last_known_addr { host: "127.2.76.193" port: 44525 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "586c1e0d9f354de9a611fa19a0955b72" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf6a5661b3cb457ea03815e04948c29b" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 44669 } health_report { overall_health: UNKNOWN } } }
I20260809 23:18:20.391780 4521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.389496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40768 (local address 127.2.76.193:44525)
0809 23:18:20.389535 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:20.389538 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:20.389569 (+ 31us) server_negotiation.cc:407] Connection header received
0809 23:18:20.389647 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:20.389648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:20.389665 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:20.389737 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:20.389831 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.390248 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.391052 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:20.391236 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:20.391299 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:20.391473 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:20.391563 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:20.391649 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:20.391674 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 23:18:20.391692 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:18:20.393174 4194 raft_consensus.cc:1261] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72 [term 1 FOLLOWER]: Refusing update from remote peer 05b1551773d14c5c91db0176b81c0d5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:20.393174 4422 raft_consensus.cc:1261] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b [term 1 FOLLOWER]: Refusing update from remote peer 05b1551773d14c5c91db0176b81c0d5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:20.393469 4537 consensus_queue.cc:1035] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf6a5661b3cb457ea03815e04948c29b" member_type: VOTER last_known_addr { host: "127.2.76.195" port: 44669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:20.393604 4557 consensus_queue.cc:1035] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "586c1e0d9f354de9a611fa19a0955b72" member_type: VOTER last_known_addr { host: "127.2.76.194" port: 34605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:20.395778 4562 mvcc.cc:205] Tried to move back new op lower bound from 7316756481608400896 to 7316756481536438272. Current Snapshot: MvccSnapshot[applied={T|T < 7316756481608400896}]
I20260809 23:18:20.395924 4566 mvcc.cc:205] Tried to move back new op lower bound from 7316756481608400896 to 7316756481536438272. Current Snapshot: MvccSnapshot[applied={T|T < 7316756481608400896}]
I20260809 23:18:20.397754 3727 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:18:20.397841 3727 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:18:20.398445 3727 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e42596c210541179ef51014ec88ce42: Sending DeleteTablet for 3 replicas of tablet 3baa40fc86804f73a219d748b99cb824
I20260809 23:18:20.398736 4160 tablet_service.cc:1514] Processing DeleteTablet for tablet 3baa40fc86804f73a219d748b99cb824 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:20 UTC) from {username='slave'} at 127.0.0.1:49524
I20260809 23:18:20.398716 3942 tablet_service.cc:1514] Processing DeleteTablet for tablet 3baa40fc86804f73a219d748b99cb824 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:20 UTC) from {username='slave'} at 127.0.0.1:40758
I20260809 23:18:20.398742 4380 tablet_service.cc:1514] Processing DeleteTablet for tablet 3baa40fc86804f73a219d748b99cb824 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:18:20 UTC) from {username='slave'} at 127.0.0.1:39868
I20260809 23:18:20.398952 4590 tablet_replica.cc:330] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b: stopping tablet replica
I20260809 23:18:20.398954 4585 tablet_replica.cc:330] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e: stopping tablet replica
I20260809 23:18:20.399036 4585 raft_consensus.cc:2229] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:18:20.399050 4590 raft_consensus.cc:2229] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:18:20.399116 4585 raft_consensus.cc:2258] T 3baa40fc86804f73a219d748b99cb824 P 05b1551773d14c5c91db0176b81c0d5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:18:20.399142 4590 raft_consensus.cc:2258] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:18:20.399415 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 3836
I20260809 23:18:20.399583 4590 ts_tablet_manager.cc:1905] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:18:20.402629 4584 tablet_replica.cc:330] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72: stopping tablet replica
I20260809 23:18:20.402647 4590 ts_tablet_manager.cc:1918] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:18:20.402721 4590 log.cc:1196] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-2/wal/wals/3baa40fc86804f73a219d748b99cb824
I20260809 23:18:20.402725 4584 raft_consensus.cc:2229] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:18:20.402822 4584 raft_consensus.cc:2258] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:18:20.402961 4590 ts_tablet_manager.cc:1939] T 3baa40fc86804f73a219d748b99cb824 P bf6a5661b3cb457ea03815e04948c29b: Deleting consensus metadata
I20260809 23:18:20.403290 3700 catalog_manager.cc:4841] TS bf6a5661b3cb457ea03815e04948c29b (127.2.76.195:44669): tablet 3baa40fc86804f73a219d748b99cb824 (table test-table [id=9389a038e6694e42bc421045c6f651e9]) successfully deleted
I20260809 23:18:20.405041 4584 ts_tablet_manager.cc:1905] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:18:20.406857 4584 ts_tablet_manager.cc:1918] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:18:20.406924 4584 log.cc:1196] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72: Deleting WAL directory at /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786317238298104-2355-0/minicluster-data/ts-1/wal/wals/3baa40fc86804f73a219d748b99cb824
I20260809 23:18:20.407169 4584 ts_tablet_manager.cc:1939] T 3baa40fc86804f73a219d748b99cb824 P 586c1e0d9f354de9a611fa19a0955b72: Deleting consensus metadata
I20260809 23:18:20.407542 3703 catalog_manager.cc:4841] TS 586c1e0d9f354de9a611fa19a0955b72 (127.2.76.194:34605): tablet 3baa40fc86804f73a219d748b99cb824 (table test-table [id=9389a038e6694e42bc421045c6f651e9]) successfully deleted
W20260809 23:18:20.411603 3701 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.76.193:44525 (error 108)
I20260809 23:18:20.412075 4517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:20.411960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.193:44525 (local address 127.0.0.1:40784)
0809 23:18:20.412014 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:20.412050 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.193:44525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 23:18:20.412212 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 4056
W20260809 23:18:20.412216 3701 catalog_manager.cc:4584] TS 05b1551773d14c5c91db0176b81c0d5e (127.2.76.193:44525): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3baa40fc86804f73a219d748b99cb824: Network error: Client connection negotiation failed: client connection to 127.2.76.193:44525: connect: Connection refused (error 111)
I20260809 23:18:20.420552 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 4282
I20260809 23:18:20.428507 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 3664
2026-08-09T23:18:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (628 ms)
[----------] 6 tests from EncryptionPolicyTest (8488 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:20 dist-test-slave-7vjv krb5kdc[4595](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:20 dist-test-slave-7vjv krb5kdc[4595](info): set up 2 sockets
Aug 09 23:18:20 dist-test-slave-7vjv krb5kdc[4595](info): commencing operation
krb5kdc: starting...
W20260809 23:18:21.571938 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:21 dist-test-slave-7vjv krb5kdc[4595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:18:21.674999 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36343
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:21.746716 4619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:21.746897 4619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:21.746932 4619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:21.747952 4619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:21.748013 4619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:21.748036 4619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:21.748071 4619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:21.748111 4619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:21.749396 4619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36343
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:21.749639 4619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:21.749838 4619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:21.753926 4631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:21.753926 4626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:21.754137 4619 server_base.cc:1034] running on GCE node
W20260809 23:18:21.754240 4629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:21.754421 4619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:21.754665 4619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:21.755791 4619 hybrid_clock.cc:648] HybridClock initialized: now 1786317501755779 us; error 26 us; skew 500 ppm
Aug 09 23:18:21 dist-test-slave-7vjv krb5kdc[4595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317501, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:21.757957 4619 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:18:21.758320 4619 webserver.cc:460] Webserver started at http://127.0.0.1:43401/ using document root <none> and password file <none>
I20260809 23:18:21.758508 4619 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:21.758565 4619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:21.758706 4619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:21.759522 4619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "943bbf4f2ed14a4f946a6908da20373c"
format_stamp: "Formatted at 2026-08-09 23:18:21 on dist-test-slave-7vjv"
server_key: "18299e2fc0e474fcac87af848b00556b"
server_key_iv: "9563408cdd0a742ee2e409ad3f5e1dfe"
server_key_version: "encryptionkey@0"
I20260809 23:18:21.759838 4619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "943bbf4f2ed14a4f946a6908da20373c"
format_stamp: "Formatted at 2026-08-09 23:18:21 on dist-test-slave-7vjv"
server_key: "18299e2fc0e474fcac87af848b00556b"
server_key_iv: "9563408cdd0a742ee2e409ad3f5e1dfe"
server_key_version: "encryptionkey@0"
I20260809 23:18:21.760993 4619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:18:21.761588 4640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:21.761736 4619 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:18:21.761818 4619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "943bbf4f2ed14a4f946a6908da20373c"
format_stamp: "Formatted at 2026-08-09 23:18:21 on dist-test-slave-7vjv"
server_key: "18299e2fc0e474fcac87af848b00556b"
server_key_iv: "9563408cdd0a742ee2e409ad3f5e1dfe"
server_key_version: "encryptionkey@0"
I20260809 23:18:21.761888 4619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:21.772506 4619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:21.773934 4619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:21.774075 4619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:21.777642 4619 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36343
I20260809 23:18:21.777685 4737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36343 every 8 connection(s)
I20260809 23:18:21.777973 4619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:21.778827 4738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:21.779078 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 4619
I20260809 23:18:21.779213 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:21.779388 2355 external_mini_cluster.cc:1428] Setting key 3203b405eace5ed686ad85aea12a7f41
I20260809 23:18:21.781148 4738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c: Bootstrap starting.
I20260809 23:18:21.781729 4738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:21.781998 4738 log.cc:824] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:21.782610 4738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c: No bootstrap required, opened a new log
I20260809 23:18:21.784417 4738 raft_consensus.cc:348] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943bbf4f2ed14a4f946a6908da20373c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36343 } }
I20260809 23:18:21.784554 4738 raft_consensus.cc:374] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:21.784610 4738 raft_consensus.cc:729] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943bbf4f2ed14a4f946a6908da20373c, State: Initialized, Role: FOLLOWER
I20260809 23:18:21.784732 4738 consensus_queue.cc:260] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943bbf4f2ed14a4f946a6908da20373c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36343 } }
I20260809 23:18:21.784817 4738 raft_consensus.cc:388] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:21.784847 4738 raft_consensus.cc:482] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:21.784886 4738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [term 0 FOLLOWER]: Advancing to term 1
Aug 09 23:18:21 dist-test-slave-7vjv krb5kdc[4595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 23:18:21.785531 4738 raft_consensus.cc:504] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943bbf4f2ed14a4f946a6908da20373c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36343 } }
I20260809 23:18:21.785635 4738 leader_election.cc:302] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 943bbf4f2ed14a4f946a6908da20373c; no voters:
I20260809 23:18:21.785802 4738 leader_election.cc:288] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:21.785846 4745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:21.785987 4745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [term 1 LEADER]: Becoming Leader. State: Replica: 943bbf4f2ed14a4f946a6908da20373c, State: Running, Role: LEADER
I20260809 23:18:21.786059 4738 sys_catalog.cc:564] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:21.786070 4745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943bbf4f2ed14a4f946a6908da20373c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36343 } }
I20260809 23:18:21.786487 4745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 943bbf4f2ed14a4f946a6908da20373c. Latest consensus state: current_term: 1 leader_uuid: "943bbf4f2ed14a4f946a6908da20373c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943bbf4f2ed14a4f946a6908da20373c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36343 } } }
I20260809 23:18:21.786585 4745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:21.786505 4748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "943bbf4f2ed14a4f946a6908da20373c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943bbf4f2ed14a4f946a6908da20373c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36343 } } }
I20260809 23:18:21.786881 4748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:21.787142 4758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:21.787580 4758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:21.789718 4743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:21.779860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55843 (local address 127.0.0.1:36343)
0809 23:18:21.780057 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:21.780065 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:21.780119 (+ 54us) server_negotiation.cc:407] Connection header received
0809 23:18:21.780474 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:21.780480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:21.780520 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:21.780644 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:21.781221 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:21.782154 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:21.782866 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:21.783215 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:21.783331 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:21.785786 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:21.785791 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:21.785791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:21.785801 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:21.788299 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:21.788704 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:21.788706 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:21.788706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:21.788774 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:21.789028 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:21.789029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:21.789029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:21.789212 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:21.789313 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:21.789468 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 23:18:21.789533 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":83,"threads_started":1}
I20260809 23:18:21.790485 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:21.793641 4758 catalog_manager.cc:1347] Generated new cluster ID: d0c9e57bcfed4a22ab60277952abd1af
I20260809 23:18:21.793721 4758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:21.798044 4743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:21.791818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42093 (local address 127.0.0.1:36343)
0809 23:18:21.791859 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:21.791860 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:21.791958 (+ 98us) server_negotiation.cc:407] Connection header received
0809 23:18:21.791996 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:21.791997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:21.792005 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:21.792053 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:21.792627 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:21.793123 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:21.794120 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:21.794430 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:21.794512 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:21.795595 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:21.795596 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:21.795596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:21.795603 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:21.796960 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:21.797315 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:21.797316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:21.797316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:21.797369 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:21.797588 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:21.797589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:21.797589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:21.797711 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:21.797794 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:21.797920 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 23:18:21.797974 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 23:18:21.807055 4758 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:21.807636 4758 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:21.809796 4758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 943bbf4f2ed14a4f946a6908da20373c: Generated new TSK 0
I20260809 23:18:21.809973 4758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:21.816212 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 4619
W20260809 23:18:21.829039 4592 connection.cc:535] client connection to 127.0.0.1:36343 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36343): Connection reset by peer (error 104)
2026-08-09T23:18:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1391 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:21 dist-test-slave-7vjv krb5kdc[4801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:21 dist-test-slave-7vjv krb5kdc[4801](info): set up 2 sockets
Aug 09 23:18:21 dist-test-slave-7vjv krb5kdc[4801](info): commencing operation
krb5kdc: starting...
W20260809 23:18:22.959534 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:23 dist-test-slave-7vjv krb5kdc[4801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:18:23.069043 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36239
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:23.141485 4826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:23.141647 4826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:23.141666 4826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:23.142673 4826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:23.142709 4826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:23.142721 4826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:23.142732 4826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:23.142743 4826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:23.143976 4826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36239
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:23.144187 4826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:23.144363 4826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:23.148524 4837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:23.148543 4834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:23.148602 4826 server_base.cc:1034] running on GCE node
W20260809 23:18:23.148566 4835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:23.148988 4826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:23.149268 4826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:23.150419 4826 hybrid_clock.cc:648] HybridClock initialized: now 1786317503150406 us; error 39 us; skew 500 ppm
Aug 09 23:18:23 dist-test-slave-7vjv krb5kdc[4801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317503, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:23.152491 4826 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:18:23.152848 4826 webserver.cc:460] Webserver started at http://127.0.0.1:33841/ using document root <none> and password file <none>
I20260809 23:18:23.153030 4826 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:23.153087 4826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:23.153213 4826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:23.154124 4826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "24a52a3f1f6e43ddbb2f9699c22393a6"
format_stamp: "Formatted at 2026-08-09 23:18:23 on dist-test-slave-7vjv"
server_key: "2d1cd34f4c8c9b47156951d6d78def72"
server_key_iv: "73ddcd8890a57b541bb3d0e1ca8eabbf"
server_key_version: "encryptionkey@0"
I20260809 23:18:23.154448 4826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "24a52a3f1f6e43ddbb2f9699c22393a6"
format_stamp: "Formatted at 2026-08-09 23:18:23 on dist-test-slave-7vjv"
server_key: "2d1cd34f4c8c9b47156951d6d78def72"
server_key_iv: "73ddcd8890a57b541bb3d0e1ca8eabbf"
server_key_version: "encryptionkey@0"
I20260809 23:18:23.155553 4826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:18:23.156179 4850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:23.156325 4826 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:23.156409 4826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "24a52a3f1f6e43ddbb2f9699c22393a6"
format_stamp: "Formatted at 2026-08-09 23:18:23 on dist-test-slave-7vjv"
server_key: "2d1cd34f4c8c9b47156951d6d78def72"
server_key_iv: "73ddcd8890a57b541bb3d0e1ca8eabbf"
server_key_version: "encryptionkey@0"
I20260809 23:18:23.156494 4826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:23.169783 4826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:23.171347 4826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:23.171482 4826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:23.175057 4826 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36239
I20260809 23:18:23.175086 4941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36239 every 8 connection(s)
I20260809 23:18:23.175361 4826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:23.176198 4943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:23.178005 4943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6: Bootstrap starting.
I20260809 23:18:23.178514 4943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:23.178766 4943 log.cc:824] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:23.179354 4943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6: No bootstrap required, opened a new log
I20260809 23:18:23.180832 4943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a52a3f1f6e43ddbb2f9699c22393a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36239 } }
I20260809 23:18:23.180949 4943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:23.180997 4943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24a52a3f1f6e43ddbb2f9699c22393a6, State: Initialized, Role: FOLLOWER
I20260809 23:18:23.181130 4943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a52a3f1f6e43ddbb2f9699c22393a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36239 } }
I20260809 23:18:23.181203 4943 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:23.181249 4943 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:23.181298 4943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:23.181785 4943 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a52a3f1f6e43ddbb2f9699c22393a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36239 } }
I20260809 23:18:23.181893 4943 leader_election.cc:302] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24a52a3f1f6e43ddbb2f9699c22393a6; no voters:
I20260809 23:18:23.182044 4943 leader_election.cc:288] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:23.182072 4948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:23.182221 4948 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [term 1 LEADER]: Becoming Leader. State: Replica: 24a52a3f1f6e43ddbb2f9699c22393a6, State: Running, Role: LEADER
I20260809 23:18:23.182261 4943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:23.182294 4948 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a52a3f1f6e43ddbb2f9699c22393a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36239 } }
I20260809 23:18:23.182598 4951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24a52a3f1f6e43ddbb2f9699c22393a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a52a3f1f6e43ddbb2f9699c22393a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36239 } } }
I20260809 23:18:23.182682 4951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:23.182770 4953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24a52a3f1f6e43ddbb2f9699c22393a6. Latest consensus state: current_term: 1 leader_uuid: "24a52a3f1f6e43ddbb2f9699c22393a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24a52a3f1f6e43ddbb2f9699c22393a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36239 } } }
I20260809 23:18:23.182840 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 4826
I20260809 23:18:23.182865 4953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:23.182951 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:23.182871 4958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:23.183148 2355 external_mini_cluster.cc:1428] Setting key 0736f96566a6b16d3f437bfcfda7c558
I20260809 23:18:23.183595 4958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:23.185164 4958 catalog_manager.cc:1347] Generated new cluster ID: 5c418b5bed5542e3b5195bbb21b79861
I20260809 23:18:23.185220 4958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:23.188143 4973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:23.183718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58461 (local address 127.0.0.1:36239)
0809 23:18:23.184298 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:23.184306 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:23.184335 (+ 29us) server_negotiation.cc:407] Connection header received
0809 23:18:23.184442 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:23.184448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:23.184509 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:23.184639 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:23.184867 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:23.185773 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:23.186455 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:23.186830 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:23.186932 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:23.187517 (+ 585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:23.187521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:23.187521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:23.187665 (+ 144us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:23.187690 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:23.187699 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:23.187779 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:23.187850 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 23:18:23.187953 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":62,"threads_started":1}
I20260809 23:18:23.188968 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:23.198335 4958 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:23.198608 4973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:23.190243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40253 (local address 127.0.0.1:36239)
0809 23:18:23.190276 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:23.190278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:23.196356 (+ 6078us) server_negotiation.cc:407] Connection header received
0809 23:18:23.196413 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:23.196414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:23.196423 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:23.196465 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:23.196602 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:23.197001 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:23.197602 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:23.197790 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:23.197852 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:23.198390 (+ 538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:23.198391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:23.198391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:23.198395 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:23.198406 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:23.198409 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:23.198458 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:23.198520 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 23:18:23.198530 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:18:23.198889 4958 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:23.208541 4958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24a52a3f1f6e43ddbb2f9699c22393a6: Generated new TSK 0
I20260809 23:18:23.208678 4958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:23.212689 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 4826
W20260809 23:18:23.233121 4797 connection.cc:535] client connection to 127.0.0.1:36239 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36239): Connection reset by peer (error 104)
2026-08-09T23:18:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:23 dist-test-slave-7vjv krb5kdc[5010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:23 dist-test-slave-7vjv krb5kdc[5010](info): set up 2 sockets
Aug 09 23:18:23 dist-test-slave-7vjv krb5kdc[5010](info): commencing operation
krb5kdc: starting...
W20260809 23:18:24.374893 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:24 dist-test-slave-7vjv krb5kdc[5010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:18:24.483659 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45019
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:24.560525 5034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:24.560705 5034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:24.560727 5034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:24.561866 5034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:24.561908 5034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:24.561921 5034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:24.561934 5034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:24.561983 5034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:24.563431 5034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45019
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:24.563661 5034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:24.563925 5034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:24.568718 5042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:24.568714 5041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:24.568747 5034 server_base.cc:1034] running on GCE node
W20260809 23:18:24.568718 5046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:24.569216 5034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:24.569526 5034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:24.570693 5034 hybrid_clock.cc:648] HybridClock initialized: now 1786317504570679 us; error 44 us; skew 500 ppm
Aug 09 23:18:24 dist-test-slave-7vjv krb5kdc[5010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317504, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:24.576329 5034 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:18:24.576769 5034 webserver.cc:460] Webserver started at http://127.0.0.1:35703/ using document root <none> and password file <none>
I20260809 23:18:24.577019 5034 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:24.577091 5034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:24.577229 5034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:24.578202 5034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "3fd730ed4bf04269ae36959c59367735"
format_stamp: "Formatted at 2026-08-09 23:18:24 on dist-test-slave-7vjv"
server_key: "befda3a894db29c7d907c2cd52fb9444"
server_key_iv: "377fb471908a4edfb55cba9c8452c63a"
server_key_version: "encryptionkey@0"
I20260809 23:18:24.578572 5034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "3fd730ed4bf04269ae36959c59367735"
format_stamp: "Formatted at 2026-08-09 23:18:24 on dist-test-slave-7vjv"
server_key: "befda3a894db29c7d907c2cd52fb9444"
server_key_iv: "377fb471908a4edfb55cba9c8452c63a"
server_key_version: "encryptionkey@0"
I20260809 23:18:24.579947 5034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:18:24.580724 5057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:24.580894 5034 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:24.581000 5034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "3fd730ed4bf04269ae36959c59367735"
format_stamp: "Formatted at 2026-08-09 23:18:24 on dist-test-slave-7vjv"
server_key: "befda3a894db29c7d907c2cd52fb9444"
server_key_iv: "377fb471908a4edfb55cba9c8452c63a"
server_key_version: "encryptionkey@0"
I20260809 23:18:24.581072 5034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:24.584255 5034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:24.586417 5034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:24.586571 5034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:24.591116 5034 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45019
I20260809 23:18:24.591148 5150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45019 every 8 connection(s)
I20260809 23:18:24.591444 5034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:24.592383 5151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:24.594388 5151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735: Bootstrap starting.
I20260809 23:18:24.594882 5151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:24.595126 5151 log.cc:824] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:24.595739 5151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735: No bootstrap required, opened a new log
I20260809 23:18:24.597256 5151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fd730ed4bf04269ae36959c59367735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45019 } }
I20260809 23:18:24.597376 5151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:24.597396 5151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fd730ed4bf04269ae36959c59367735, State: Initialized, Role: FOLLOWER
I20260809 23:18:24.597520 5151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fd730ed4bf04269ae36959c59367735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45019 } }
I20260809 23:18:24.597594 5151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:24.597615 5151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:24.597661 5151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:24.597823 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 5034
I20260809 23:18:24.597922 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:24.598114 2355 external_mini_cluster.cc:1428] Setting key 94d78982bef103edf32de8e778d1be6e
I20260809 23:18:24.598312 5151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fd730ed4bf04269ae36959c59367735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45019 } }
I20260809 23:18:24.598421 5151 leader_election.cc:302] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fd730ed4bf04269ae36959c59367735; no voters:
I20260809 23:18:24.598582 5151 leader_election.cc:288] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:24.598734 5159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:24.598807 5151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:24.598861 5159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [term 1 LEADER]: Becoming Leader. State: Replica: 3fd730ed4bf04269ae36959c59367735, State: Running, Role: LEADER
I20260809 23:18:24.598949 5159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fd730ed4bf04269ae36959c59367735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45019 } }
I20260809 23:18:24.599273 5159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fd730ed4bf04269ae36959c59367735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fd730ed4bf04269ae36959c59367735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45019 } } }
I20260809 23:18:24.599651 5159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:24.599793 5159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fd730ed4bf04269ae36959c59367735. Latest consensus state: current_term: 1 leader_uuid: "3fd730ed4bf04269ae36959c59367735" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fd730ed4bf04269ae36959c59367735" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45019 } } }
I20260809 23:18:24.599855 5159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:24.599987 5175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:24.600234 5158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:24.598511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56515 (local address 127.0.0.1:45019)
0809 23:18:24.598707 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:24.598715 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:24.598743 (+ 28us) server_negotiation.cc:407] Connection header received
0809 23:18:24.598846 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:24.598852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:24.598879 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:24.599029 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:24.599679 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:24.599683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:24.599683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:24.599686 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:24.599711 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:24.599723 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:24.599827 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:24.599952 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 23:18:24.599997 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":62,"threads_started":1}
I20260809 23:18:24.600524 5175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:24.601187 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:24.602005 5175 catalog_manager.cc:1347] Generated new cluster ID: c26e6fb5b3d14435bdb3327ef9a27430
I20260809 23:18:24.602057 5175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:24.603597 5158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:24.602681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48817 (local address 127.0.0.1:45019)
0809 23:18:24.602715 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:24.602717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:24.602757 (+ 40us) server_negotiation.cc:407] Connection header received
0809 23:18:24.602809 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:24.602810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:24.602814 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:24.602883 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:24.603388 (+ 505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:24.603389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:24.603390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:24.603394 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:24.603408 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:24.603412 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:24.603477 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:24.603527 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 23:18:24.603539 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:18:24.607904 5175 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:24.608044 5175 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:24.613090 5175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fd730ed4bf04269ae36959c59367735: Generated new TSK 0
I20260809 23:18:24.613255 5175 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:24.617477 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 5034
2026-08-09T23:18:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:24 dist-test-slave-7vjv krb5kdc[5200](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:24 dist-test-slave-7vjv krb5kdc[5200](info): set up 2 sockets
Aug 09 23:18:24 dist-test-slave-7vjv krb5kdc[5200](info): commencing operation
krb5kdc: starting...
W20260809 23:18:25.770598 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:25 dist-test-slave-7vjv krb5kdc[5200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:18:25.874598 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41311
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:25.947350 5226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:25.947510 5226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:25.947528 5226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:25.948645 5226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:25.948686 5226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:25.948700 5226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:25.948714 5226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:25.948725 5226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:25.950024 5226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:25.950237 5226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:25.950407 5226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:25.954582 5232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:25.954579 5226 server_base.cc:1034] running on GCE node
W20260809 23:18:25.954597 5235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:25.954591 5233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:25.955091 5226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:25.955343 5226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:25.956497 5226 hybrid_clock.cc:648] HybridClock initialized: now 1786317505956484 us; error 32 us; skew 500 ppm
Aug 09 23:18:25 dist-test-slave-7vjv krb5kdc[5200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317505, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:25.958768 5226 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:18:25.959161 5226 webserver.cc:460] Webserver started at http://127.0.0.1:36993/ using document root <none> and password file <none>
I20260809 23:18:25.959394 5226 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:25.959467 5226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:25.959638 5226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:25.960543 5226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "bc1cff956976450b97c7ada096dd3edc"
format_stamp: "Formatted at 2026-08-09 23:18:25 on dist-test-slave-7vjv"
server_key: "ca0ccb54c5e497b264dac5fecb0c3428"
server_key_iv: "083689b891bbf771f38bef8e7422f628"
server_key_version: "encryptionkey@0"
I20260809 23:18:25.960845 5226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "bc1cff956976450b97c7ada096dd3edc"
format_stamp: "Formatted at 2026-08-09 23:18:25 on dist-test-slave-7vjv"
server_key: "ca0ccb54c5e497b264dac5fecb0c3428"
server_key_iv: "083689b891bbf771f38bef8e7422f628"
server_key_version: "encryptionkey@0"
I20260809 23:18:25.962076 5226 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:18:25.962757 5250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:25.962906 5226 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:25.962994 5226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "bc1cff956976450b97c7ada096dd3edc"
format_stamp: "Formatted at 2026-08-09 23:18:25 on dist-test-slave-7vjv"
server_key: "ca0ccb54c5e497b264dac5fecb0c3428"
server_key_iv: "083689b891bbf771f38bef8e7422f628"
server_key_version: "encryptionkey@0"
I20260809 23:18:25.963063 5226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:25.974022 5226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:25.976274 5226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:25.976429 5226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:25.980414 5226 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41311
I20260809 23:18:25.980445 5334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41311 every 8 connection(s)
I20260809 23:18:25.980749 5226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:25.981572 5337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:25.983467 5337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc: Bootstrap starting.
I20260809 23:18:25.983979 5337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:25.984324 5337 log.cc:824] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:25.984964 5337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc: No bootstrap required, opened a new log
I20260809 23:18:25.986357 5337 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc1cff956976450b97c7ada096dd3edc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41311 } }
I20260809 23:18:25.986469 5337 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:25.986524 5337 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc1cff956976450b97c7ada096dd3edc, State: Initialized, Role: FOLLOWER
I20260809 23:18:25.986671 5337 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc1cff956976450b97c7ada096dd3edc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41311 } }
I20260809 23:18:25.986727 5337 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:25.986745 5337 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:25.986806 5337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:25.987260 5337 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc1cff956976450b97c7ada096dd3edc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41311 } }
I20260809 23:18:25.987358 5337 leader_election.cc:302] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc1cff956976450b97c7ada096dd3edc; no voters:
I20260809 23:18:25.987520 5337 leader_election.cc:288] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:25.987561 5346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:25.987721 5346 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [term 1 LEADER]: Becoming Leader. State: Replica: bc1cff956976450b97c7ada096dd3edc, State: Running, Role: LEADER
I20260809 23:18:25.987735 5337 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:25.987872 5346 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc1cff956976450b97c7ada096dd3edc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41311 } }
I20260809 23:18:25.988293 5351 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc1cff956976450b97c7ada096dd3edc. Latest consensus state: current_term: 1 leader_uuid: "bc1cff956976450b97c7ada096dd3edc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc1cff956976450b97c7ada096dd3edc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41311 } } }
I20260809 23:18:25.988361 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 5226
I20260809 23:18:25.988399 5348 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc1cff956976450b97c7ada096dd3edc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc1cff956976450b97c7ada096dd3edc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41311 } } }
I20260809 23:18:25.988428 5351 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:25.988449 5348 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:25.988484 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:25.988678 2355 external_mini_cluster.cc:1428] Setting key e026e17eefcebd984ef0efd4e1261e02
I20260809 23:18:25.989337 5363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:25.989925 5363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:25.992571 5363 catalog_manager.cc:1347] Generated new cluster ID: 8dc1fc9f32334f93915ce81955307b36
I20260809 23:18:25.992620 5363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:18:25 dist-test-slave-7vjv krb5kdc[5200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 23:18:25.999603 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:25.989353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38457 (local address 127.0.0.1:41311)
0809 23:18:25.989550 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:25.989558 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:25.989587 (+ 29us) server_negotiation.cc:407] Connection header received
0809 23:18:25.989681 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:25.989686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:25.989720 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:25.989861 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:25.990873 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:25.992053 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:25.992987 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:25.993566 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:25.993673 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:25.996408 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:25.996411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:25.996412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:25.996422 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:25.998129 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:25.998507 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:25.998510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:25.998510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:25.998625 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:25.998821 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:25.998822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:25.998822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:25.999073 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:25.999201 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:25.999303 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 23:18:25.999422 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":56,"threads_started":1}
I20260809 23:18:26.000067 5363 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:26.000658 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:26.000759 5363 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:26.007601 5363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc1cff956976450b97c7ada096dd3edc: Generated new TSK 0
I20260809 23:18:26.007788 5363 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:26.008428 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:26.002052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34413 (local address 127.0.0.1:41311)
0809 23:18:26.002082 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:26.002084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:26.002146 (+ 62us) server_negotiation.cc:407] Connection header received
0809 23:18:26.002200 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:26.002201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:26.002209 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:26.002281 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:26.002859 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:26.003571 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:26.004529 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:26.004799 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:26.004852 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:26.005952 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:26.005953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:26.005953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:26.005959 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:26.007371 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:26.007725 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:26.007726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:26.007726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:26.007779 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:26.007979 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:26.007979 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:26.007980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:26.008128 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:26.008210 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:26.008283 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 23:18:26.008353 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 23:18:26.010197 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 5226
2026-08-09T23:18:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:26 dist-test-slave-7vjv krb5kdc[5390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:26 dist-test-slave-7vjv krb5kdc[5390](info): set up 2 sockets
Aug 09 23:18:26 dist-test-slave-7vjv krb5kdc[5390](info): commencing operation
krb5kdc: starting...
W20260809 23:18:27.199542 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:27 dist-test-slave-7vjv krb5kdc[5390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:18:27.320966 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38759
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:27.393894 5411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:27.394073 5411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:27.394093 5411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:27.395100 5411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:27.395138 5411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:27.395151 5411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:27.395162 5411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:27.395205 5411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:27.396483 5411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:27.396636 5411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:27.396836 5411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:27.401175 5417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:27.401151 5423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:27.401194 5421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:27.401365 5411 server_base.cc:1034] running on GCE node
I20260809 23:18:27.401582 5411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:27.401823 5411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:27.402974 5411 hybrid_clock.cc:648] HybridClock initialized: now 1786317507402955 us; error 31 us; skew 500 ppm
Aug 09 23:18:27 dist-test-slave-7vjv krb5kdc[5390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317507, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:27.405129 5411 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:18:27.405478 5411 webserver.cc:460] Webserver started at http://127.0.0.1:38901/ using document root <none> and password file <none>
I20260809 23:18:27.405665 5411 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:27.405733 5411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:27.405862 5411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:27.406706 5411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "5b253bfe9aff47d0922125237fe0a443"
format_stamp: "Formatted at 2026-08-09 23:18:27 on dist-test-slave-7vjv"
server_key: "53c642830efb7c132b6e8807cee9af7d"
server_key_iv: "8d7ce10c8cbfe62f62647b744e6eb0e8"
server_key_version: "encryptionkey@0"
I20260809 23:18:27.407020 5411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "5b253bfe9aff47d0922125237fe0a443"
format_stamp: "Formatted at 2026-08-09 23:18:27 on dist-test-slave-7vjv"
server_key: "53c642830efb7c132b6e8807cee9af7d"
server_key_iv: "8d7ce10c8cbfe62f62647b744e6eb0e8"
server_key_version: "encryptionkey@0"
I20260809 23:18:27.408164 5411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:18:27.408797 5437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:27.408974 5411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:27.409072 5411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "5b253bfe9aff47d0922125237fe0a443"
format_stamp: "Formatted at 2026-08-09 23:18:27 on dist-test-slave-7vjv"
server_key: "53c642830efb7c132b6e8807cee9af7d"
server_key_iv: "8d7ce10c8cbfe62f62647b744e6eb0e8"
server_key_version: "encryptionkey@0"
I20260809 23:18:27.409164 5411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:27.422147 5411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:27.423666 5411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:27.423810 5411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:27.427664 5411 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38759
I20260809 23:18:27.427693 5533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38759 every 8 connection(s)
I20260809 23:18:27.427984 5411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:27.428855 5534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:27.430845 5534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443: Bootstrap starting.
I20260809 23:18:27.431329 5534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:27.431542 5534 log.cc:824] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:27.432243 5534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443: No bootstrap required, opened a new log
I20260809 23:18:27.433622 5534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b253bfe9aff47d0922125237fe0a443" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38759 } }
I20260809 23:18:27.433744 5534 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:27.433763 5534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b253bfe9aff47d0922125237fe0a443, State: Initialized, Role: FOLLOWER
I20260809 23:18:27.433848 5534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b253bfe9aff47d0922125237fe0a443" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38759 } }
I20260809 23:18:27.433890 5534 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:27.433915 5534 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:27.433938 5534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:27.434327 5534 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b253bfe9aff47d0922125237fe0a443" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38759 } }
I20260809 23:18:27.434401 5534 leader_election.cc:302] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b253bfe9aff47d0922125237fe0a443; no voters:
I20260809 23:18:27.434531 5534 leader_election.cc:288] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:27.434561 5537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:27.434669 5537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [term 1 LEADER]: Becoming Leader. State: Replica: 5b253bfe9aff47d0922125237fe0a443, State: Running, Role: LEADER
I20260809 23:18:27.434758 5534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:27.434794 5537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b253bfe9aff47d0922125237fe0a443" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38759 } }
I20260809 23:18:27.434916 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 5411
I20260809 23:18:27.435048 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:27.435225 2355 external_mini_cluster.cc:1428] Setting key 79ec68a924d156390144a22de4c38557
I20260809 23:18:27.435376 5539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b253bfe9aff47d0922125237fe0a443. Latest consensus state: current_term: 1 leader_uuid: "5b253bfe9aff47d0922125237fe0a443" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b253bfe9aff47d0922125237fe0a443" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38759 } } }
I20260809 23:18:27.435484 5539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:27.435678 5548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:27.436144 5538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b253bfe9aff47d0922125237fe0a443" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b253bfe9aff47d0922125237fe0a443" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38759 } } }
I20260809 23:18:27.436221 5538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:27.436203 5548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:27.438884 5548 catalog_manager.cc:1347] Generated new cluster ID: a031ee1e86464e0c940df09ef8f0139c
I20260809 23:18:27.438934 5548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:27.441135 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:27.435785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48133 (local address 127.0.0.1:38759)
0809 23:18:27.436288 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:27.436296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:27.436545 (+ 249us) server_negotiation.cc:407] Connection header received
0809 23:18:27.436721 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:27.436726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:27.436823 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:27.436933 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:27.437237 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:27.438431 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:27.439288 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:27.439710 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:27.439829 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:27.440539 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:27.440542 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:27.440543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:27.440701 (+ 158us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:27.440728 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:27.440737 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:27.440808 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:27.440876 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 23:18:27.440932 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":67,"threads_started":1}
I20260809 23:18:27.441942 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:27.446559 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:27.443190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40783 (local address 127.0.0.1:38759)
0809 23:18:27.443222 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:27.443224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:27.444282 (+ 1058us) server_negotiation.cc:407] Connection header received
0809 23:18:27.444349 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:27.444350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:27.444360 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:27.444402 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:27.444521 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:27.444945 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:27.445574 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:27.445751 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:27.445817 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:27.446346 (+ 529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:27.446348 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:27.446348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:27.446352 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:27.446362 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:27.446366 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:27.446406 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:27.446454 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 23:18:27.446472 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:18:27.458689 5548 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:27.459363 5548 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:27.464697 5548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b253bfe9aff47d0922125237fe0a443: Generated new TSK 0
I20260809 23:18:27.464847 5548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:27.467959 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 5411
2026-08-09T23:18:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1460 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:27 dist-test-slave-7vjv krb5kdc[5581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:27 dist-test-slave-7vjv krb5kdc[5581](info): set up 2 sockets
Aug 09 23:18:27 dist-test-slave-7vjv krb5kdc[5581](info): commencing operation
krb5kdc: starting...
W20260809 23:18:28.614722 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:28 dist-test-slave-7vjv krb5kdc[5581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:18:28.720337 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36051
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:28.790725 5604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:28.790906 5604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:28.790941 5604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:28.791926 5604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:28.791987 5604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:28.792011 5604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:28.792028 5604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:28.792057 5604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:28.793356 5604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:28.793604 5604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:28.793797 5604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:28.798046 5617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:28.798022 5612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:28.798139 5611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:28.798421 5604 server_base.cc:1034] running on GCE node
I20260809 23:18:28.798633 5604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:28.798851 5604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:28.800072 5604 hybrid_clock.cc:648] HybridClock initialized: now 1786317508800048 us; error 28 us; skew 500 ppm
Aug 09 23:18:28 dist-test-slave-7vjv krb5kdc[5581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317508, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:28.802345 5604 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:18:28.802696 5604 webserver.cc:460] Webserver started at http://127.0.0.1:42371/ using document root <none> and password file <none>
I20260809 23:18:28.802901 5604 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:28.802939 5604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:28.803056 5604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:28.803989 5604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "2a43fcb5d27349adbcac3229359aa4e3"
format_stamp: "Formatted at 2026-08-09 23:18:28 on dist-test-slave-7vjv"
server_key: "5dab29c82da0c57b5923fe969b42432d"
server_key_iv: "00fbf1672f5d8bfb082cf2c777a8e9e3"
server_key_version: "encryptionkey@0"
I20260809 23:18:28.804376 5604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "2a43fcb5d27349adbcac3229359aa4e3"
format_stamp: "Formatted at 2026-08-09 23:18:28 on dist-test-slave-7vjv"
server_key: "5dab29c82da0c57b5923fe969b42432d"
server_key_iv: "00fbf1672f5d8bfb082cf2c777a8e9e3"
server_key_version: "encryptionkey@0"
I20260809 23:18:28.805588 5604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:18:28.806248 5630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:28.806409 5604 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:28.806501 5604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "2a43fcb5d27349adbcac3229359aa4e3"
format_stamp: "Formatted at 2026-08-09 23:18:28 on dist-test-slave-7vjv"
server_key: "5dab29c82da0c57b5923fe969b42432d"
server_key_iv: "00fbf1672f5d8bfb082cf2c777a8e9e3"
server_key_version: "encryptionkey@0"
I20260809 23:18:28.806577 5604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:28.809633 5604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:28.811141 5604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:28.811273 5604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:28.814733 5604 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36051
I20260809 23:18:28.814762 5709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36051 every 8 connection(s)
I20260809 23:18:28.815177 5604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:28.816187 5710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:28.818073 5710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3: Bootstrap starting.
I20260809 23:18:28.818630 5710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:28.818907 5710 log.cc:824] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:28.819536 5710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3: No bootstrap required, opened a new log
I20260809 23:18:28.821125 5710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a43fcb5d27349adbcac3229359aa4e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36051 } }
I20260809 23:18:28.821240 5710 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:28.821292 5710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a43fcb5d27349adbcac3229359aa4e3, State: Initialized, Role: FOLLOWER
I20260809 23:18:28.821437 5710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a43fcb5d27349adbcac3229359aa4e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36051 } }
I20260809 23:18:28.821494 5710 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:28.821516 5710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:28.821578 5710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:28.822041 5710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a43fcb5d27349adbcac3229359aa4e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36051 } }
I20260809 23:18:28.822144 5710 leader_election.cc:302] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a43fcb5d27349adbcac3229359aa4e3; no voters:
I20260809 23:18:28.822309 5710 leader_election.cc:288] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:28.822359 5713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:28.822556 5710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:28.822597 5713 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [term 1 LEADER]: Becoming Leader. State: Replica: 2a43fcb5d27349adbcac3229359aa4e3, State: Running, Role: LEADER
I20260809 23:18:28.822707 5713 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a43fcb5d27349adbcac3229359aa4e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36051 } }
I20260809 23:18:28.823253 5718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a43fcb5d27349adbcac3229359aa4e3. Latest consensus state: current_term: 1 leader_uuid: "2a43fcb5d27349adbcac3229359aa4e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a43fcb5d27349adbcac3229359aa4e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36051 } } }
I20260809 23:18:28.823264 5716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a43fcb5d27349adbcac3229359aa4e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a43fcb5d27349adbcac3229359aa4e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36051 } } }
I20260809 23:18:28.823343 5718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:28.823400 5716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:28.823608 5729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:28.824265 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 5604
I20260809 23:18:28.824381 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:28.824546 2355 external_mini_cluster.cc:1428] Setting key 778103e2078aef517309d4bcb1686907
I20260809 23:18:28.824812 5729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:28.826314 5729 catalog_manager.cc:1347] Generated new cluster ID: c5d6768521c94f18b8fbe7842524562c
I20260809 23:18:28.826367 5729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:28.829407 5742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:28.824938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56039 (local address 127.0.0.1:36051)
0809 23:18:28.825253 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:28.825261 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:28.828270 (+ 3009us) server_negotiation.cc:407] Connection header received
0809 23:18:28.828392 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:28.828396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:28.828416 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:28.828536 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:28.829029 (+ 493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:28.829032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:28.829032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:28.829036 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:28.829068 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:28.829081 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:28.829147 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:28.829212 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 23:18:28.829249 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":87,"threads_started":1}
I20260809 23:18:28.830176 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:28.832310 5742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:28.831372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36115 (local address 127.0.0.1:36051)
0809 23:18:28.831422 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:28.831423 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:28.831558 (+ 135us) server_negotiation.cc:407] Connection header received
0809 23:18:28.831600 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:28.831601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:28.831605 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:28.831648 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:28.832114 (+ 466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:28.832115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:28.832115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:28.832119 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:28.832130 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:28.832133 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:28.832171 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:28.832227 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 23:18:28.832239 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 23:18:28.843693 5729 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:28.843853 5729 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:28.848377 5729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a43fcb5d27349adbcac3229359aa4e3: Generated new TSK 0
I20260809 23:18:28.848526 5729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:28.854758 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 5604
2026-08-09T23:18:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1388 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:28 dist-test-slave-7vjv krb5kdc[5756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:28 dist-test-slave-7vjv krb5kdc[5756](info): set up 2 sockets
Aug 09 23:18:28 dist-test-slave-7vjv krb5kdc[5756](info): commencing operation
krb5kdc: starting...
W20260809 23:18:30.013429 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:30 dist-test-slave-7vjv krb5kdc[5756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:18:30.116732 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44135
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:30.188251 5785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:30.188405 5785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:30.188423 5785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:30.189404 5785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:30.189441 5785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:30.189453 5785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:30.189464 5785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:30.189510 5785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:30.190871 5785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:30.191097 5785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:30.191299 5785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:30.195631 5795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:30.195742 5792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:30.195713 5785 server_base.cc:1034] running on GCE node
W20260809 23:18:30.195657 5793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:30.196019 5785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:30.196285 5785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:30.197443 5785 hybrid_clock.cc:648] HybridClock initialized: now 1786317510197441 us; error 41 us; skew 500 ppm
Aug 09 23:18:30 dist-test-slave-7vjv krb5kdc[5756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317510, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:30.203367 5785 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:18:30.203685 5785 webserver.cc:460] Webserver started at http://127.0.0.1:36649/ using document root <none> and password file <none>
I20260809 23:18:30.203850 5785 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:30.203882 5785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:30.203953 5785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:30.204854 5785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "129e0783ec564ed0a68f4ecfaa98d4b0"
format_stamp: "Formatted at 2026-08-09 23:18:30 on dist-test-slave-7vjv"
server_key: "da7e97feb286d58c75bc02f2e5408914"
server_key_iv: "e7b4665a2bce410ba45b6e8ae31eaa92"
server_key_version: "encryptionkey@0"
I20260809 23:18:30.205117 5785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "129e0783ec564ed0a68f4ecfaa98d4b0"
format_stamp: "Formatted at 2026-08-09 23:18:30 on dist-test-slave-7vjv"
server_key: "da7e97feb286d58c75bc02f2e5408914"
server_key_iv: "e7b4665a2bce410ba45b6e8ae31eaa92"
server_key_version: "encryptionkey@0"
I20260809 23:18:30.206147 5785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:18:30.206717 5810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:30.206851 5785 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:18:30.206919 5785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "129e0783ec564ed0a68f4ecfaa98d4b0"
format_stamp: "Formatted at 2026-08-09 23:18:30 on dist-test-slave-7vjv"
server_key: "da7e97feb286d58c75bc02f2e5408914"
server_key_iv: "e7b4665a2bce410ba45b6e8ae31eaa92"
server_key_version: "encryptionkey@0"
I20260809 23:18:30.206965 5785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:30.214113 5785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:30.215510 5785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:30.215602 5785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:30.218959 5785 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44135
I20260809 23:18:30.219002 5891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44135 every 8 connection(s)
I20260809 23:18:30.219285 5785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:30.220163 5892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:30.220580 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 5785
I20260809 23:18:30.220688 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:30.220863 2355 external_mini_cluster.cc:1428] Setting key f054bdd498acffa65f9628d8cf6aa33e
I20260809 23:18:30.222702 5892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0: Bootstrap starting.
I20260809 23:18:30.223270 5892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:30.223560 5892 log.cc:824] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:30.224238 5892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0: No bootstrap required, opened a new log
I20260809 23:18:30.226037 5892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "129e0783ec564ed0a68f4ecfaa98d4b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44135 } }
I20260809 23:18:30.226145 5892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:30.226163 5892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 129e0783ec564ed0a68f4ecfaa98d4b0, State: Initialized, Role: FOLLOWER
I20260809 23:18:30.226272 5892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "129e0783ec564ed0a68f4ecfaa98d4b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44135 } }
I20260809 23:18:30.226336 5892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:30.226356 5892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:30.226413 5892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 23:18:30 dist-test-slave-7vjv krb5kdc[5756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 23:18:30.226835 5892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "129e0783ec564ed0a68f4ecfaa98d4b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44135 } }
I20260809 23:18:30.226926 5892 leader_election.cc:302] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 129e0783ec564ed0a68f4ecfaa98d4b0; no voters:
I20260809 23:18:30.227087 5892 leader_election.cc:288] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:30.227131 5900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:30.227250 5900 raft_consensus.cc:686] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [term 1 LEADER]: Becoming Leader. State: Replica: 129e0783ec564ed0a68f4ecfaa98d4b0, State: Running, Role: LEADER
I20260809 23:18:30.227320 5900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "129e0783ec564ed0a68f4ecfaa98d4b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44135 } }
I20260809 23:18:30.227344 5892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:30.228567 5903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 129e0783ec564ed0a68f4ecfaa98d4b0. Latest consensus state: current_term: 1 leader_uuid: "129e0783ec564ed0a68f4ecfaa98d4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "129e0783ec564ed0a68f4ecfaa98d4b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44135 } } }
I20260809 23:18:30.228657 5903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:30.228888 5919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:30.229305 5902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "129e0783ec564ed0a68f4ecfaa98d4b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "129e0783ec564ed0a68f4ecfaa98d4b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44135 } } }
I20260809 23:18:30.229368 5919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:30.229384 5902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:30.230728 5919 catalog_manager.cc:1347] Generated new cluster ID: 04bc9a05f9064b9a90f982864376b592
I20260809 23:18:30.230777 5919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:30.231499 5898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:30.221306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:34233 (local address 127.0.0.1:44135)
0809 23:18:30.221466 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:30.221474 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:30.221501 (+ 27us) server_negotiation.cc:407] Connection header received
0809 23:18:30.221924 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:30.221932 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:30.221984 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:30.222128 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:30.222614 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:30.223782 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:30.224571 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:30.225104 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:30.225196 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:30.227393 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:30.227397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:30.227397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:30.227407 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:30.229662 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:30.230028 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:30.230031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:30.230031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:30.230120 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:30.230350 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:30.230351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:30.230351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:30.230618 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:30.230750 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:30.231209 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 23:18:30.231325 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260809 23:18:30.232384 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:30.240494 5898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:30.233744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:55317 (local address 127.0.0.1:44135)
0809 23:18:30.233775 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:30.233776 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:30.233881 (+ 105us) server_negotiation.cc:407] Connection header received
0809 23:18:30.233958 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:30.233959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:30.233967 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:30.233990 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:30.234577 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:30.235242 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:30.236238 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:30.236515 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:30.236553 (+ 38us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:30.237799 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:30.237800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:30.237800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:30.237806 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:30.239220 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:30.239639 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:30.239640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:30.239641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:30.239694 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:30.239918 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:30.239918 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:30.239919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:30.240044 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:30.240139 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:30.240358 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 23:18:30.240415 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 23:18:30.247901 5919 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:30.248391 5919 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:30.257269 5919 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 129e0783ec564ed0a68f4ecfaa98d4b0: Generated new TSK 0
I20260809 23:18:30.257424 5919 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:30.261514 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 5785
W20260809 23:18:30.281356 5753 connection.cc:535] client connection to 127.0.0.1:44135 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44135): Connection reset by peer (error 104)
2026-08-09T23:18:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:30 dist-test-slave-7vjv krb5kdc[5944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:30 dist-test-slave-7vjv krb5kdc[5944](info): set up 2 sockets
Aug 09 23:18:30 dist-test-slave-7vjv krb5kdc[5944](info): commencing operation
krb5kdc: starting...
W20260809 23:18:31.444308 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:31 dist-test-slave-7vjv krb5kdc[5944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:18:31.550235 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43355
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:31.621583 5970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:31.621743 5970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:31.621762 5970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:31.622645 5970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:31.622675 5970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:31.622687 5970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:31.622699 5970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:31.622709 5970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:31.623908 5970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:31.624078 5970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:31.624287 5970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:31.628264 5976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:31.628269 5985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:31.628283 5975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:31.628440 5970 server_base.cc:1034] running on GCE node
I20260809 23:18:31.628635 5970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:31.628878 5970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:31.630014 5970 hybrid_clock.cc:648] HybridClock initialized: now 1786317511630004 us; error 26 us; skew 500 ppm
Aug 09 23:18:31 dist-test-slave-7vjv krb5kdc[5944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317511, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:31.632200 5970 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:18:31.632634 5970 webserver.cc:460] Webserver started at http://127.0.0.1:38855/ using document root <none> and password file <none>
I20260809 23:18:31.632833 5970 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:31.632895 5970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:31.633010 5970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:31.633867 5970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "d802e257f1704d80b7e9dc21bfe7bb68"
format_stamp: "Formatted at 2026-08-09 23:18:31 on dist-test-slave-7vjv"
server_key: "66c3b48a0692b15440aebda565f0fd0d"
server_key_iv: "83248bd725d27720e3150012badd88a7"
server_key_version: "encryptionkey@0"
I20260809 23:18:31.634198 5970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "d802e257f1704d80b7e9dc21bfe7bb68"
format_stamp: "Formatted at 2026-08-09 23:18:31 on dist-test-slave-7vjv"
server_key: "66c3b48a0692b15440aebda565f0fd0d"
server_key_iv: "83248bd725d27720e3150012badd88a7"
server_key_version: "encryptionkey@0"
I20260809 23:18:31.635324 5970 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:18:31.635931 5997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:31.636080 5970 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:31.636190 5970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "d802e257f1704d80b7e9dc21bfe7bb68"
format_stamp: "Formatted at 2026-08-09 23:18:31 on dist-test-slave-7vjv"
server_key: "66c3b48a0692b15440aebda565f0fd0d"
server_key_iv: "83248bd725d27720e3150012badd88a7"
server_key_version: "encryptionkey@0"
I20260809 23:18:31.636266 5970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:31.648412 5970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:31.649941 5970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:31.650097 5970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:31.653970 5970 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43355
I20260809 23:18:31.653999 6076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43355 every 8 connection(s)
I20260809 23:18:31.654376 5970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:31.655257 6077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:31.657189 6077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68: Bootstrap starting.
I20260809 23:18:31.657617 6077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:31.657833 6077 log.cc:824] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:31.658385 6077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68: No bootstrap required, opened a new log
I20260809 23:18:31.659708 6077 raft_consensus.cc:348] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d802e257f1704d80b7e9dc21bfe7bb68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43355 } }
I20260809 23:18:31.659825 6077 raft_consensus.cc:374] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:31.659848 6077 raft_consensus.cc:729] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d802e257f1704d80b7e9dc21bfe7bb68, State: Initialized, Role: FOLLOWER
I20260809 23:18:31.659941 6077 consensus_queue.cc:260] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d802e257f1704d80b7e9dc21bfe7bb68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43355 } }
I20260809 23:18:31.659989 6077 raft_consensus.cc:388] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:31.660015 6077 raft_consensus.cc:482] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:31.660039 6077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:31.660523 6077 raft_consensus.cc:504] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d802e257f1704d80b7e9dc21bfe7bb68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43355 } }
I20260809 23:18:31.660602 6077 leader_election.cc:302] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d802e257f1704d80b7e9dc21bfe7bb68; no voters:
I20260809 23:18:31.660709 6077 leader_election.cc:288] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:31.660755 6081 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:31.660899 6081 raft_consensus.cc:686] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [term 1 LEADER]: Becoming Leader. State: Replica: d802e257f1704d80b7e9dc21bfe7bb68, State: Running, Role: LEADER
I20260809 23:18:31.660949 6077 sys_catalog.cc:564] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:31.661011 6081 consensus_queue.cc:237] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d802e257f1704d80b7e9dc21bfe7bb68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43355 } }
I20260809 23:18:31.661388 6085 sys_catalog.cc:455] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d802e257f1704d80b7e9dc21bfe7bb68. Latest consensus state: current_term: 1 leader_uuid: "d802e257f1704d80b7e9dc21bfe7bb68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d802e257f1704d80b7e9dc21bfe7bb68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43355 } } }
I20260809 23:18:31.661464 6085 sys_catalog.cc:458] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:31.661517 6082 sys_catalog.cc:455] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d802e257f1704d80b7e9dc21bfe7bb68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d802e257f1704d80b7e9dc21bfe7bb68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43355 } } }
I20260809 23:18:31.661597 6082 sys_catalog.cc:458] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:31.662245 6090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:31.662802 6090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:31.664145 6090 catalog_manager.cc:1347] Generated new cluster ID: 6c4ceb75614a4d989157289754d3a2b5
I20260809 23:18:31.664196 6090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:31.664472 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 5970
I20260809 23:18:31.664561 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:31.664712 2355 external_mini_cluster.cc:1428] Setting key 4ce99ea02cb89b7e6a84978f4fdad727
I20260809 23:18:31.680696 6090 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:31.681089 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:31.665077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:50021 (local address 127.0.0.1:43355)
0809 23:18:31.665211 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:31.665218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:31.676287 (+ 11069us) server_negotiation.cc:407] Connection header received
0809 23:18:31.676416 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:31.676422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:31.676471 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:31.676595 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:31.676800 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:31.678009 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:31.678913 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:31.679435 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:31.679538 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:31.680265 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:31.680269 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:31.680269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:31.680595 (+ 326us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:31.680630 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:31.680644 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:31.680721 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:31.680870 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 23:18:31.680895 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":64,"threads_started":1}
I20260809 23:18:31.681419 6090 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:31.681983 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:31.686573 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:31.683468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:46899 (local address 127.0.0.1:43355)
0809 23:18:31.683495 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:31.683497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:31.683504 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:18:31.683565 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:31.683565 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:31.683573 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:31.683619 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:31.683853 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:31.684520 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:31.685451 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:31.685704 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:31.685754 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:31.686364 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:31.686365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:31.686366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:31.686370 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:31.686383 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:31.686387 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:31.686432 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:31.686505 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 23:18:31.686514 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260809 23:18:31.687094 6090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d802e257f1704d80b7e9dc21bfe7bb68: Generated new TSK 0
I20260809 23:18:31.687259 6090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:31.691973 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 5970
W20260809 23:18:31.700834 5941 connection.cc:535] client connection to 127.0.0.1:43355 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43355): Connection reset by peer (error 104)
2026-08-09T23:18:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1419 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:31 dist-test-slave-7vjv krb5kdc[6136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:31 dist-test-slave-7vjv krb5kdc[6136](info): set up 2 sockets
Aug 09 23:18:31 dist-test-slave-7vjv krb5kdc[6136](info): commencing operation
krb5kdc: starting...
W20260809 23:18:32.851099 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:32 dist-test-slave-7vjv krb5kdc[6136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:18:32.956238 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34509
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:33.028059 6160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:33.028229 6160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:33.028249 6160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:33.029214 6160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:33.029251 6160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:33.029263 6160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:33.029294 6160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:33.029320 6160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:33.030604 6160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:33.030827 6160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:33.031054 6160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:33.035403 6172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:33.035415 6171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:33.035635 6176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:33.035673 6160 server_base.cc:1034] running on GCE node
I20260809 23:18:33.035941 6160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:33.036278 6160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:33.037456 6160 hybrid_clock.cc:648] HybridClock initialized: now 1786317513037422 us; error 43 us; skew 500 ppm
Aug 09 23:18:33 dist-test-slave-7vjv krb5kdc[6136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317513, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:33.040606 6160 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:18:33.043164 6160 webserver.cc:460] Webserver started at http://127.0.0.1:35129/ using document root <none> and password file <none>
I20260809 23:18:33.043411 6160 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:33.043501 6160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:33.043668 6160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:33.044872 6160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "38a15534e2714552a63e684d6078bf4d"
format_stamp: "Formatted at 2026-08-09 23:18:33 on dist-test-slave-7vjv"
server_key: "ebd306a071de6846eb7bcc929632caea"
server_key_iv: "773bd816c77d296c1da83e511aa6ae41"
server_key_version: "encryptionkey@0"
I20260809 23:18:33.045192 6160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "38a15534e2714552a63e684d6078bf4d"
format_stamp: "Formatted at 2026-08-09 23:18:33 on dist-test-slave-7vjv"
server_key: "ebd306a071de6846eb7bcc929632caea"
server_key_iv: "773bd816c77d296c1da83e511aa6ae41"
server_key_version: "encryptionkey@0"
I20260809 23:18:33.046360 6160 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:18:33.047097 6187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:33.047295 6160 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:18:33.047395 6160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "38a15534e2714552a63e684d6078bf4d"
format_stamp: "Formatted at 2026-08-09 23:18:33 on dist-test-slave-7vjv"
server_key: "ebd306a071de6846eb7bcc929632caea"
server_key_iv: "773bd816c77d296c1da83e511aa6ae41"
server_key_version: "encryptionkey@0"
I20260809 23:18:33.047482 6160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:33.050370 6160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:33.051980 6160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:33.052160 6160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:33.055945 6160 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34509
I20260809 23:18:33.055977 6277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34509 every 8 connection(s)
I20260809 23:18:33.056310 6160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:33.057144 6279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:33.059096 6279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d: Bootstrap starting.
I20260809 23:18:33.059629 6279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:33.059828 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 6160
I20260809 23:18:33.059944 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:33.059955 6279 log.cc:824] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:33.060176 2355 external_mini_cluster.cc:1428] Setting key c1f92c8a5bf4426cc151e6b8bc18e0c0
I20260809 23:18:33.060748 6279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d: No bootstrap required, opened a new log
I20260809 23:18:33.062351 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:33.060618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:50885 (local address 127.0.0.1:34509)
0809 23:18:33.060785 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:33.060793 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:33.060833 (+ 40us) server_negotiation.cc:407] Connection header received
0809 23:18:33.061259 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:33.061267 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:33.061295 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:33.061439 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:33.061967 (+ 528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:33.061970 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:33.061971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:33.061975 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:33.062014 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:33.062026 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:33.062120 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:33.062150 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 23:18:33.062185 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":65,"threads_started":1}
I20260809 23:18:33.063426 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:33.063437 6279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a15534e2714552a63e684d6078bf4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34509 } }
I20260809 23:18:33.063639 6279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:33.063690 6279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a15534e2714552a63e684d6078bf4d, State: Initialized, Role: FOLLOWER
I20260809 23:18:33.063812 6279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a15534e2714552a63e684d6078bf4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34509 } }
I20260809 23:18:33.063915 6279 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:33.063964 6279 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:33.064018 6279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:33.064627 6279 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a15534e2714552a63e684d6078bf4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34509 } }
I20260809 23:18:33.064749 6279 leader_election.cc:302] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38a15534e2714552a63e684d6078bf4d; no voters:
I20260809 23:18:33.064909 6279 leader_election.cc:288] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:33.064939 6295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:33.065084 6295 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [term 1 LEADER]: Becoming Leader. State: Replica: 38a15534e2714552a63e684d6078bf4d, State: Running, Role: LEADER
I20260809 23:18:33.065171 6279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:33.065161 6295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a15534e2714552a63e684d6078bf4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34509 } }
I20260809 23:18:33.065508 6297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38a15534e2714552a63e684d6078bf4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a15534e2714552a63e684d6078bf4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34509 } } }
I20260809 23:18:33.065610 6297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:33.065780 6299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38a15534e2714552a63e684d6078bf4d. Latest consensus state: current_term: 1 leader_uuid: "38a15534e2714552a63e684d6078bf4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a15534e2714552a63e684d6078bf4d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34509 } } }
I20260809 23:18:33.065865 6299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:33.066313 6308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:33.066735 6308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:33.068029 6288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:33.064888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:33393 (local address 127.0.0.1:34509)
0809 23:18:33.064925 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:33.064927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:33.065557 (+ 630us) server_negotiation.cc:407] Connection header received
0809 23:18:33.065654 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:33.065655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:33.065658 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:33.065685 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:33.067821 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:33.067822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:33.067823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:33.067827 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:33.067839 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:33.067844 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:33.067910 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:33.067968 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 23:18:33.067979 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:18:33.068210 6308 catalog_manager.cc:1347] Generated new cluster ID: 19f9cfc25b6848248496dbae1951bdee
I20260809 23:18:33.068252 6308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:33.091497 6308 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:33.091640 6308 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:33.098847 6308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38a15534e2714552a63e684d6078bf4d: Generated new TSK 0
I20260809 23:18:33.099030 6308 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:33.105589 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 6160
2026-08-09T23:18:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1421 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:33 dist-test-slave-7vjv krb5kdc[6340](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:33 dist-test-slave-7vjv krb5kdc[6340](info): set up 2 sockets
Aug 09 23:18:33 dist-test-slave-7vjv krb5kdc[6340](info): commencing operation
krb5kdc: starting...
W20260809 23:18:34.257576 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:34 dist-test-slave-7vjv krb5kdc[6340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:18:34.360783 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38595
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:34.432139 6364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:34.432309 6364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:34.432330 6364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:34.433530 6364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:34.433590 6364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:34.433606 6364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:34.433619 6364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:34.433631 6364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:34.435240 6364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38595
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:34.435488 6364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:34.435709 6364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:34.440197 6376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:34.440235 6364 server_base.cc:1034] running on GCE node
W20260809 23:18:34.440212 6372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:34.440215 6371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:34.440681 6364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:34.440959 6364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:34.442107 6364 hybrid_clock.cc:648] HybridClock initialized: now 1786317514442101 us; error 34 us; skew 500 ppm
Aug 09 23:18:34 dist-test-slave-7vjv krb5kdc[6340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317514, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:34.444339 6364 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:18:34.444657 6364 webserver.cc:460] Webserver started at http://127.0.0.1:43553/ using document root <none> and password file <none>
I20260809 23:18:34.444813 6364 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:34.444849 6364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:34.444923 6364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:34.445719 6364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "2160955931a94b2cb3c5041beb324674"
format_stamp: "Formatted at 2026-08-09 23:18:34 on dist-test-slave-7vjv"
server_key: "9b97b0054bbe71f31f78402665a77b11"
server_key_iv: "9abae0d9a15d099250c2d96ca2804ea5"
server_key_version: "encryptionkey@0"
I20260809 23:18:34.446099 6364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "2160955931a94b2cb3c5041beb324674"
format_stamp: "Formatted at 2026-08-09 23:18:34 on dist-test-slave-7vjv"
server_key: "9b97b0054bbe71f31f78402665a77b11"
server_key_iv: "9abae0d9a15d099250c2d96ca2804ea5"
server_key_version: "encryptionkey@0"
I20260809 23:18:34.447265 6364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:18:34.447907 6388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:34.448163 6364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:18:34.448246 6364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "2160955931a94b2cb3c5041beb324674"
format_stamp: "Formatted at 2026-08-09 23:18:34 on dist-test-slave-7vjv"
server_key: "9b97b0054bbe71f31f78402665a77b11"
server_key_iv: "9abae0d9a15d099250c2d96ca2804ea5"
server_key_version: "encryptionkey@0"
I20260809 23:18:34.448308 6364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:34.463083 6364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:34.464710 6364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:34.464846 6364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:34.468603 6364 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38595
I20260809 23:18:34.468667 6484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38595 every 8 connection(s)
I20260809 23:18:34.469012 6364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:34.469856 6485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:34.471735 6485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674: Bootstrap starting.
I20260809 23:18:34.472290 6485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:34.472537 6485 log.cc:824] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:34.473126 6485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674: No bootstrap required, opened a new log
I20260809 23:18:34.474602 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 6364
I20260809 23:18:34.474713 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:34.474656 6485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2160955931a94b2cb3c5041beb324674" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38595 } }
I20260809 23:18:34.474819 6485 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:34.474859 6485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2160955931a94b2cb3c5041beb324674, State: Initialized, Role: FOLLOWER
I20260809 23:18:34.474898 2355 external_mini_cluster.cc:1428] Setting key b1bd9a2f61945bd935526a0c4f8d513b
I20260809 23:18:34.474988 6485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2160955931a94b2cb3c5041beb324674" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38595 } }
I20260809 23:18:34.475092 6485 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:34.475131 6485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:34.475191 6485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:34.475966 6485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2160955931a94b2cb3c5041beb324674" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38595 } }
I20260809 23:18:34.476058 6485 leader_election.cc:302] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2160955931a94b2cb3c5041beb324674; no voters:
I20260809 23:18:34.476284 6485 leader_election.cc:288] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:34.476303 6492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:34.476408 6492 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [term 1 LEADER]: Becoming Leader. State: Replica: 2160955931a94b2cb3c5041beb324674, State: Running, Role: LEADER
I20260809 23:18:34.476545 6485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:34.476558 6492 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2160955931a94b2cb3c5041beb324674" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38595 } }
I20260809 23:18:34.476907 6492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2160955931a94b2cb3c5041beb324674. Latest consensus state: current_term: 1 leader_uuid: "2160955931a94b2cb3c5041beb324674" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2160955931a94b2cb3c5041beb324674" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38595 } } }
I20260809 23:18:34.476925 6496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2160955931a94b2cb3c5041beb324674" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2160955931a94b2cb3c5041beb324674" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38595 } } }
I20260809 23:18:34.477123 6496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:34.477046 6492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:34.478104 6508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:34.478549 6508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:34.480448 6508 catalog_manager.cc:1347] Generated new cluster ID: be27ece959d0427f90e87caf333ef02f
I20260809 23:18:34.480494 6508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:18:34 dist-test-slave-7vjv krb5kdc[6340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 23:18:34.485574 6508 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:34.485810 6491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:34.475329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:51237 (local address 127.0.0.1:38595)
0809 23:18:34.475525 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:34.475532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:34.475571 (+ 39us) server_negotiation.cc:407] Connection header received
0809 23:18:34.475657 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:34.475662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:34.475698 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:34.475821 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:34.476552 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:34.477970 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:34.479019 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:34.479517 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:34.479600 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:34.482195 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:34.482198 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:34.482199 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:34.482209 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:34.483916 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:34.484373 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:34.484376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:34.484376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:34.484478 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:34.484708 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:34.484709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:34.484709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:34.484961 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:34.485112 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:34.485521 (+ 409us) server_negotiation.cc:299] Negotiation successful
0809 23:18:34.485680 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260809 23:18:34.486160 6508 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:34.486616 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:34.493976 6491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:34.487893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:58937 (local address 127.0.0.1:38595)
0809 23:18:34.487927 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:34.487928 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:34.488265 (+ 337us) server_negotiation.cc:407] Connection header received
0809 23:18:34.488328 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:34.488328 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:34.488341 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:34.488390 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:34.488922 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:34.489446 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:34.490169 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:34.490397 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:34.490473 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:34.491572 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:34.491573 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:34.491574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:34.491580 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:34.492893 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:34.493234 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:34.493235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:34.493235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:34.493282 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:34.493523 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:34.493524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:34.493525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:34.493622 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:34.493721 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:34.493836 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 23:18:34.493889 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 23:18:34.501361 6508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2160955931a94b2cb3c5041beb324674: Generated new TSK 0
I20260809 23:18:34.501504 6508 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:34.504689 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 6364
W20260809 23:18:34.520622 6337 connection.cc:535] client connection to 127.0.0.1:38595 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38595): Connection reset by peer (error 104)
2026-08-09T23:18:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:34 dist-test-slave-7vjv krb5kdc[6540](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:34 dist-test-slave-7vjv krb5kdc[6540](info): set up 2 sockets
Aug 09 23:18:34 dist-test-slave-7vjv krb5kdc[6540](info): commencing operation
krb5kdc: starting...
W20260809 23:18:35.629769 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:35 dist-test-slave-7vjv krb5kdc[6540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:18:35.734745 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33819
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33819
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:35.807196 6559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:35.807389 6559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:35.807426 6559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:35.808408 6559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:35.808468 6559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:35.808496 6559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:35.808525 6559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:35.808547 6559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:35.809803 6559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33819
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:35.810060 6559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:35.810261 6559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:35.814508 6566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:35.814599 6567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:35.814605 6559 server_base.cc:1034] running on GCE node
W20260809 23:18:35.814512 6570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:35.814960 6559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:35.815264 6559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:35.816480 6559 hybrid_clock.cc:648] HybridClock initialized: now 1786317515816465 us; error 37 us; skew 500 ppm
Aug 09 23:18:35 dist-test-slave-7vjv krb5kdc[6540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317515, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:35.821812 6559 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:18:35.822365 6559 webserver.cc:460] Webserver started at http://127.0.0.1:34455/ using document root <none> and password file <none>
I20260809 23:18:35.822604 6559 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:35.822660 6559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:35.822781 6559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:35.823896 6559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "8c3d81daed674c008476e6d4262d4d4c"
format_stamp: "Formatted at 2026-08-09 23:18:35 on dist-test-slave-7vjv"
server_key: "aeb1365b390495916d9671880a6176de"
server_key_iv: "a17e0f5bbf2768f32fef3e408281b339"
server_key_version: "encryptionkey@0"
I20260809 23:18:35.824270 6559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "8c3d81daed674c008476e6d4262d4d4c"
format_stamp: "Formatted at 2026-08-09 23:18:35 on dist-test-slave-7vjv"
server_key: "aeb1365b390495916d9671880a6176de"
server_key_iv: "a17e0f5bbf2768f32fef3e408281b339"
server_key_version: "encryptionkey@0"
I20260809 23:18:35.825569 6559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:18:35.826217 6581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:35.826371 6559 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:18:35.826464 6559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "8c3d81daed674c008476e6d4262d4d4c"
format_stamp: "Formatted at 2026-08-09 23:18:35 on dist-test-slave-7vjv"
server_key: "aeb1365b390495916d9671880a6176de"
server_key_iv: "a17e0f5bbf2768f32fef3e408281b339"
server_key_version: "encryptionkey@0"
I20260809 23:18:35.826535 6559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:35.836331 6559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:35.838336 6559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:35.838485 6559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:35.842237 6559 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33819
I20260809 23:18:35.842263 6664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33819 every 8 connection(s)
I20260809 23:18:35.842572 6559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:35.843309 6666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:35.845256 6666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c: Bootstrap starting.
I20260809 23:18:35.845719 6666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:35.845932 6666 log.cc:824] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:35.846499 6666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c: No bootstrap required, opened a new log
I20260809 23:18:35.848008 6666 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3d81daed674c008476e6d4262d4d4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33819 } }
I20260809 23:18:35.848198 6666 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:35.848222 6666 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d81daed674c008476e6d4262d4d4c, State: Initialized, Role: FOLLOWER
I20260809 23:18:35.848309 6666 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3d81daed674c008476e6d4262d4d4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33819 } }
I20260809 23:18:35.848353 6666 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:35.848371 6666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:35.848398 6666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:35.848809 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 6559
I20260809 23:18:35.848918 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:35.848906 6666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3d81daed674c008476e6d4262d4d4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33819 } }
I20260809 23:18:35.848992 6666 leader_election.cc:302] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c3d81daed674c008476e6d4262d4d4c; no voters:
I20260809 23:18:35.849131 2355 external_mini_cluster.cc:1428] Setting key 849b1c71132ebfbb47bc5ba2204b5cf4
I20260809 23:18:35.849176 6666 leader_election.cc:288] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:35.849246 6672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:35.849391 6666 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:35.849431 6672 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [term 1 LEADER]: Becoming Leader. State: Replica: 8c3d81daed674c008476e6d4262d4d4c, State: Running, Role: LEADER
I20260809 23:18:35.849602 6672 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3d81daed674c008476e6d4262d4d4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33819 } }
I20260809 23:18:35.850128 6673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c3d81daed674c008476e6d4262d4d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3d81daed674c008476e6d4262d4d4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33819 } } }
I20260809 23:18:35.850216 6673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:35.850531 6674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c3d81daed674c008476e6d4262d4d4c. Latest consensus state: current_term: 1 leader_uuid: "8c3d81daed674c008476e6d4262d4d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c3d81daed674c008476e6d4262d4d4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33819 } } }
I20260809 23:18:35.850627 6674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c [sys.catalog]: This master's current role is: LEADER
W20260809 23:18:35.851343 6708 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:18:35.851397 6708 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:18:35.851447 6691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:35.851573 6691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:35.853114 6691 catalog_manager.cc:1347] Generated new cluster ID: d6728eceb44d45f29d6d54a97c39e0ba
I20260809 23:18:35.853174 6691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:35.854864 6680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:35.849653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:37143 (local address 127.0.0.1:33819)
0809 23:18:35.849803 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:35.849810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:35.850072 (+ 262us) server_negotiation.cc:407] Connection header received
0809 23:18:35.850302 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:35.850309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:35.850371 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:35.850491 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:35.850972 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:35.851980 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:35.852941 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:35.853480 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:35.853608 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:35.854262 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:35.854265 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:35.854265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:35.854424 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:35.854451 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:35.854460 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:35.854561 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:35.854635 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 23:18:35.854659 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260809 23:18:35.855611 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:35.859548 6680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:35.856992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:35645 (local address 127.0.0.1:33819)
0809 23:18:35.857051 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:35.857052 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:35.857095 (+ 43us) server_negotiation.cc:407] Connection header received
0809 23:18:35.857155 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:35.857156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:35.857166 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:35.857212 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:35.857339 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:35.857737 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:35.858338 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:35.858542 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:35.858610 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:35.859196 (+ 586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:35.859197 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:35.859197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:35.859201 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:35.859213 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:35.859217 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:35.859265 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:35.859459 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 23:18:35.859471 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:18:35.863601 6691 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:35.864244 6691 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:35.867265 6691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c3d81daed674c008476e6d4262d4d4c: Generated new TSK 0
I20260809 23:18:35.867415 6691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:35.868975 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 6559
W20260809 23:18:35.881453 6533 connection.cc:535] client connection to 127.0.0.1:33819 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33819): Connection reset by peer (error 104)
2026-08-09T23:18:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1360 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:35 dist-test-slave-7vjv krb5kdc[6724](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:35 dist-test-slave-7vjv krb5kdc[6724](info): set up 2 sockets
Aug 09 23:18:35 dist-test-slave-7vjv krb5kdc[6724](info): commencing operation
krb5kdc: starting...
W20260809 23:18:36.985112 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:37 dist-test-slave-7vjv krb5kdc[6724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:18:37.088323 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32959
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:37.159699 6746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:37.159894 6746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:37.159933 6746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:37.160954 6746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:37.161019 6746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:37.161042 6746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:37.161064 6746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:37.161090 6746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:37.162349 6746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32959
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:37.162590 6746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:37.162784 6746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:37.166939 6757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:37.166940 6758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:37.166978 6763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:37.167088 6746 server_base.cc:1034] running on GCE node
I20260809 23:18:37.167452 6746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:37.167701 6746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:37.168867 6746 hybrid_clock.cc:648] HybridClock initialized: now 1786317517168855 us; error 39 us; skew 500 ppm
Aug 09 23:18:37 dist-test-slave-7vjv krb5kdc[6724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317517, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:37.171083 6746 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:18:37.171540 6746 webserver.cc:460] Webserver started at http://127.0.0.1:42171/ using document root <none> and password file <none>
I20260809 23:18:37.171751 6746 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:37.171813 6746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:37.171960 6746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:37.172911 6746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "7b1d74a9819f404b802ee4a8f9678697"
format_stamp: "Formatted at 2026-08-09 23:18:37 on dist-test-slave-7vjv"
server_key: "df93d550bf332096c4ccd511772b1c37"
server_key_iv: "0126d04e4b34ecafde3375b74e741879"
server_key_version: "encryptionkey@0"
I20260809 23:18:37.173241 6746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "7b1d74a9819f404b802ee4a8f9678697"
format_stamp: "Formatted at 2026-08-09 23:18:37 on dist-test-slave-7vjv"
server_key: "df93d550bf332096c4ccd511772b1c37"
server_key_iv: "0126d04e4b34ecafde3375b74e741879"
server_key_version: "encryptionkey@0"
I20260809 23:18:37.174449 6746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:18:37.175089 6771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:37.175243 6746 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:37.175329 6746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "7b1d74a9819f404b802ee4a8f9678697"
format_stamp: "Formatted at 2026-08-09 23:18:37 on dist-test-slave-7vjv"
server_key: "df93d550bf332096c4ccd511772b1c37"
server_key_iv: "0126d04e4b34ecafde3375b74e741879"
server_key_version: "encryptionkey@0"
I20260809 23:18:37.175405 6746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:37.178143 6746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:37.180064 6746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:37.180233 6746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:37.183847 6746 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32959
I20260809 23:18:37.183877 6868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32959 every 8 connection(s)
I20260809 23:18:37.184190 6746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:37.184908 6869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:37.186794 6869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697: Bootstrap starting.
I20260809 23:18:37.187312 6869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:37.187588 6869 log.cc:824] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:37.188237 6869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697: No bootstrap required, opened a new log
I20260809 23:18:37.189546 6869 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1d74a9819f404b802ee4a8f9678697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32959 } }
I20260809 23:18:37.189657 6869 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:37.189707 6869 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b1d74a9819f404b802ee4a8f9678697, State: Initialized, Role: FOLLOWER
I20260809 23:18:37.189841 6869 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1d74a9819f404b802ee4a8f9678697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32959 } }
I20260809 23:18:37.189898 6869 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:37.189916 6869 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:37.189975 6869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:37.190439 6869 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1d74a9819f404b802ee4a8f9678697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32959 } }
I20260809 23:18:37.190560 6869 leader_election.cc:302] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b1d74a9819f404b802ee4a8f9678697; no voters:
I20260809 23:18:37.190737 6869 leader_election.cc:288] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:37.190753 6874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:37.190907 6874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [term 1 LEADER]: Becoming Leader. State: Replica: 7b1d74a9819f404b802ee4a8f9678697, State: Running, Role: LEADER
I20260809 23:18:37.190960 6869 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:37.191057 6874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1d74a9819f404b802ee4a8f9678697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32959 } }
I20260809 23:18:37.191403 6876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b1d74a9819f404b802ee4a8f9678697. Latest consensus state: current_term: 1 leader_uuid: "7b1d74a9819f404b802ee4a8f9678697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1d74a9819f404b802ee4a8f9678697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32959 } } }
I20260809 23:18:37.191489 6876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:37.191509 6875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b1d74a9819f404b802ee4a8f9678697" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b1d74a9819f404b802ee4a8f9678697" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32959 } } }
I20260809 23:18:37.191563 6875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:37.191850 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 6746
I20260809 23:18:37.191959 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:37.192209 2355 external_mini_cluster.cc:1428] Setting key f5b9ff7a95190abceee6ff3b5d01361d
I20260809 23:18:37.192202 6888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:37.193177 6888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:37.194440 6888 catalog_manager.cc:1347] Generated new cluster ID: cc8644d026024631a53266b8ebf4c69a
I20260809 23:18:37.194491 6888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:37.196956 6902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:37.192589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:42645 (local address 127.0.0.1:32959)
0809 23:18:37.192782 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:37.192790 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:37.192838 (+ 48us) server_negotiation.cc:407] Connection header received
0809 23:18:37.192934 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:37.192940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:37.192967 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:37.193099 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:37.196535 (+ 3436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:37.196539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:37.196539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:37.196543 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:37.196598 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:37.196607 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:37.196671 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:37.196746 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 23:18:37.196787 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260809 23:18:37.197777 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:37.201196 6902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:37.199025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:54253 (local address 127.0.0.1:32959)
0809 23:18:37.199058 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:37.199060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:37.200258 (+ 1198us) server_negotiation.cc:407] Connection header received
0809 23:18:37.200319 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:37.200320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:37.200324 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:37.200388 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:37.200943 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:37.200944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:37.200944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:37.200948 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:37.200959 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:37.200963 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:37.201030 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:37.201099 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 23:18:37.201112 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:18:37.215231 6888 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:37.215368 6888 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:37.220976 6888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b1d74a9819f404b802ee4a8f9678697: Generated new TSK 0
I20260809 23:18:37.221191 6888 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:37.222388 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 6746
2026-08-09T23:18:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1354 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:37 dist-test-slave-7vjv krb5kdc[6918](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:37 dist-test-slave-7vjv krb5kdc[6918](info): set up 2 sockets
Aug 09 23:18:37 dist-test-slave-7vjv krb5kdc[6918](info): commencing operation
krb5kdc: starting...
W20260809 23:18:38.359375 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:38 dist-test-slave-7vjv krb5kdc[6918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:18:38.462271 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:37803
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:37803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:38.537019 6939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:38.537237 6939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:38.537268 6939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:38.538337 6939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:38.538375 6939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:38.538391 6939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:38.538405 6939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:38.538434 6939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:38.539866 6939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:37803
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:37803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:38.540038 6939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:38.540202 6939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:38.544363 6945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:38.544443 6953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:38.544387 6947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:38.544420 6939 server_base.cc:1034] running on GCE node
I20260809 23:18:38.544730 6939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:38.544997 6939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:38.546155 6939 hybrid_clock.cc:648] HybridClock initialized: now 1786317518546142 us; error 25 us; skew 500 ppm
Aug 09 23:18:38 dist-test-slave-7vjv krb5kdc[6918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317518, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:38.548657 6939 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:18:38.549181 6939 webserver.cc:460] Webserver started at http://127.2.76.254:46657/ using document root <none> and password file <none>
I20260809 23:18:38.549365 6939 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:38.549409 6939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:38.549510 6939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:38.550448 6939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "1131814d40b343f1bc30afc525b72146"
format_stamp: "Formatted at 2026-08-09 23:18:38 on dist-test-slave-7vjv"
server_key: "51e4790d8b164bf5a59e22eb467aef87"
server_key_iv: "be9e410353f71beecb857be9ca5e5886"
server_key_version: "encryptionkey@0"
I20260809 23:18:38.550748 6939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "1131814d40b343f1bc30afc525b72146"
format_stamp: "Formatted at 2026-08-09 23:18:38 on dist-test-slave-7vjv"
server_key: "51e4790d8b164bf5a59e22eb467aef87"
server_key_iv: "be9e410353f71beecb857be9ca5e5886"
server_key_version: "encryptionkey@0"
I20260809 23:18:38.551971 6939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:18:38.552726 6966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:38.552886 6939 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:38.552989 6939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "1131814d40b343f1bc30afc525b72146"
format_stamp: "Formatted at 2026-08-09 23:18:38 on dist-test-slave-7vjv"
server_key: "51e4790d8b164bf5a59e22eb467aef87"
server_key_iv: "be9e410353f71beecb857be9ca5e5886"
server_key_version: "encryptionkey@0"
I20260809 23:18:38.553084 6939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:38.561219 6939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:38.563145 6939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:38.563283 6939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:38.567181 6939 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:37803
I20260809 23:18:38.567211 7044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:37803 every 8 connection(s)
I20260809 23:18:38.567499 6939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:38.568269 7045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:38.570153 7045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146: Bootstrap starting.
I20260809 23:18:38.570669 7045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:38.570928 7045 log.cc:824] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:38.571557 7045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146: No bootstrap required, opened a new log
I20260809 23:18:38.573033 7045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1131814d40b343f1bc30afc525b72146" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37803 } }
I20260809 23:18:38.573145 7045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:38.573165 7045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1131814d40b343f1bc30afc525b72146, State: Initialized, Role: FOLLOWER
I20260809 23:18:38.573326 7045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1131814d40b343f1bc30afc525b72146" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37803 } }
I20260809 23:18:38.573380 7045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:38.573426 7045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:38.573472 7045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:38.574206 7045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1131814d40b343f1bc30afc525b72146" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37803 } }
I20260809 23:18:38.574311 7045 leader_election.cc:302] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1131814d40b343f1bc30afc525b72146; no voters:
I20260809 23:18:38.574476 7045 leader_election.cc:288] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:38.574510 7052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:38.574666 7052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [term 1 LEADER]: Becoming Leader. State: Replica: 1131814d40b343f1bc30afc525b72146, State: Running, Role: LEADER
I20260809 23:18:38.574714 7045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:38.574786 7052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1131814d40b343f1bc30afc525b72146" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37803 } }
I20260809 23:18:38.575153 7055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1131814d40b343f1bc30afc525b72146. Latest consensus state: current_term: 1 leader_uuid: "1131814d40b343f1bc30afc525b72146" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1131814d40b343f1bc30afc525b72146" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37803 } } }
I20260809 23:18:38.575253 7055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:38.575150 7053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1131814d40b343f1bc30afc525b72146" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1131814d40b343f1bc30afc525b72146" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 37803 } } }
I20260809 23:18:38.575485 7053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:38.575771 7068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:38.576267 7068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:38.576624 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 6939
I20260809 23:18:38.576725 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:38.576926 2355 external_mini_cluster.cc:1428] Setting key 7bce5327a13c61df8fb408c16c50c5ad
I20260809 23:18:38.577917 7068 catalog_manager.cc:1347] Generated new cluster ID: bcea22c4fbdb413a859d9527b28f50f4
I20260809 23:18:38.577970 7068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:18:38 dist-test-slave-7vjv krb5kdc[6918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:38.589130 7068 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:38.589766 7068 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:38.590679 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:38.577327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37059 (local address 127.2.76.254:37803)
0809 23:18:38.580412 (+ 3085us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:38.580420 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:38.580448 (+ 28us) server_negotiation.cc:407] Connection header received
0809 23:18:38.580553 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:38.580559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:38.580613 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:38.580719 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:38.581448 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:38.582694 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:38.583790 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:38.584382 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:38.584464 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:38.587090 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:38.587094 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:38.587094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:38.587105 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:38.588972 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:38.589421 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:38.589424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:38.589425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:38.589517 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:38.589762 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:38.589763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:38.589763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:38.590027 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:38.590204 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:38.590321 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 23:18:38.590517 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3027,"thread_start_us":59,"threads_started":1}
I20260809 23:18:38.591472 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:38.593653 7068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1131814d40b343f1bc30afc525b72146: Generated new TSK 0
I20260809 23:18:38.593792 7068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:38.601847 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:38.592746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54155 (local address 127.2.76.254:37803)
0809 23:18:38.592780 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:38.592781 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:38.596280 (+ 3499us) server_negotiation.cc:407] Connection header received
0809 23:18:38.596344 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:38.596345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:38.596353 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:38.596393 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:38.596861 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:38.597301 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:38.597944 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:38.598159 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:38.598233 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:38.599344 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:38.599346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:38.599346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:38.599353 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:38.600716 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:38.601055 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:38.601056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:38.601057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:38.601112 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:38.601326 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:38.601327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:38.601328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:38.601447 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:38.601553 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:38.601702 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 23:18:38.601762 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:18:38.603591 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 6939
W20260809 23:18:38.614821 6916 connection.cc:535] client connection to 127.2.76.254:37803 recv error: Network error: failed to read from TLS socket (remote: 127.2.76.254:37803): Connection reset by peer (error 104)
2026-08-09T23:18:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1378 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:38 dist-test-slave-7vjv krb5kdc[7105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:38 dist-test-slave-7vjv krb5kdc[7105](info): set up 2 sockets
Aug 09 23:18:38 dist-test-slave-7vjv krb5kdc[7105](info): commencing operation
krb5kdc: starting...
W20260809 23:18:39.720911 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:39 dist-test-slave-7vjv krb5kdc[7105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:18:39.822479 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:38777
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:38777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:39.893494 7132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:39.893648 7132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:39.893667 7132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:39.894690 7132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:39.894729 7132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:39.894742 7132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:39.894752 7132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:39.894796 7132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:39.896052 7132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:38777
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:38777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:39.896317 7132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:39.896515 7132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:39.900585 7141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:39.900559 7152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:39.900552 7140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:39.900800 7132 server_base.cc:1034] running on GCE node
I20260809 23:18:39.901028 7132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:39.901281 7132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:39.902453 7132 hybrid_clock.cc:648] HybridClock initialized: now 1786317519902443 us; error 44 us; skew 500 ppm
Aug 09 23:18:39 dist-test-slave-7vjv krb5kdc[7105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317519, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:39.904584 7132 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:18:39.904896 7132 webserver.cc:460] Webserver started at http://127.2.76.254:43479/ using document root <none> and password file <none>
I20260809 23:18:39.905071 7132 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:39.905105 7132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:39.905175 7132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:39.906028 7132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "c2121316cd78455cba5ac959e72aa312"
format_stamp: "Formatted at 2026-08-09 23:18:39 on dist-test-slave-7vjv"
server_key: "141493a758ef258a71bd67d64c123680"
server_key_iv: "cf4c01d877dcdf6a458aa2f1d5da6258"
server_key_version: "encryptionkey@0"
I20260809 23:18:39.906296 7132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "c2121316cd78455cba5ac959e72aa312"
format_stamp: "Formatted at 2026-08-09 23:18:39 on dist-test-slave-7vjv"
server_key: "141493a758ef258a71bd67d64c123680"
server_key_iv: "cf4c01d877dcdf6a458aa2f1d5da6258"
server_key_version: "encryptionkey@0"
I20260809 23:18:39.907343 7132 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 23:18:39.907907 7168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:39.908072 7132 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:39.908195 7132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "c2121316cd78455cba5ac959e72aa312"
format_stamp: "Formatted at 2026-08-09 23:18:39 on dist-test-slave-7vjv"
server_key: "141493a758ef258a71bd67d64c123680"
server_key_iv: "cf4c01d877dcdf6a458aa2f1d5da6258"
server_key_version: "encryptionkey@0"
I20260809 23:18:39.908269 7132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:39.934734 7132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:39.936424 7132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:39.936561 7132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:39.939929 7132 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:38777
I20260809 23:18:39.939980 7249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:38777 every 8 connection(s)
I20260809 23:18:39.940306 7132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:39.941063 7250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:39.942935 7250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312: Bootstrap starting.
I20260809 23:18:39.943441 7250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:39.943702 7250 log.cc:824] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:39.944314 7250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312: No bootstrap required, opened a new log
I20260809 23:18:39.945678 7250 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2121316cd78455cba5ac959e72aa312" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38777 } }
I20260809 23:18:39.945792 7250 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:39.945860 7250 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2121316cd78455cba5ac959e72aa312, State: Initialized, Role: FOLLOWER
I20260809 23:18:39.946010 7250 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2121316cd78455cba5ac959e72aa312" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38777 } }
I20260809 23:18:39.946069 7250 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:39.946103 7250 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:39.946149 7250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:39.946659 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 7132
I20260809 23:18:39.946780 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:39.946976 2355 external_mini_cluster.cc:1428] Setting key 3e3eb98d72c50fa05b974dfc66381caa
I20260809 23:18:39.947122 7250 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2121316cd78455cba5ac959e72aa312" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38777 } }
I20260809 23:18:39.947233 7250 leader_election.cc:302] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2121316cd78455cba5ac959e72aa312; no voters:
I20260809 23:18:39.947384 7250 leader_election.cc:288] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:39.947404 7255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:39.947567 7255 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [term 1 LEADER]: Becoming Leader. State: Replica: c2121316cd78455cba5ac959e72aa312, State: Running, Role: LEADER
I20260809 23:18:39.947659 7255 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2121316cd78455cba5ac959e72aa312" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38777 } }
I20260809 23:18:39.947716 7250 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:39.947914 7255 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2121316cd78455cba5ac959e72aa312. Latest consensus state: current_term: 1 leader_uuid: "c2121316cd78455cba5ac959e72aa312" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2121316cd78455cba5ac959e72aa312" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38777 } } }
I20260809 23:18:39.947923 7258 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2121316cd78455cba5ac959e72aa312" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2121316cd78455cba5ac959e72aa312" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38777 } } }
I20260809 23:18:39.947991 7255 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:39.948001 7258 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:39.948616 7264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:39.949075 7264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:39.950434 7264 catalog_manager.cc:1347] Generated new cluster ID: 49b76a688e73445483085d1adc878caa
I20260809 23:18:39.950498 7264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:39.952757 7257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:39.947379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41535 (local address 127.2.76.254:38777)
0809 23:18:39.948240 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:39.948246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:39.948268 (+ 22us) server_negotiation.cc:407] Connection header received
0809 23:18:39.948382 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:39.948387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:39.948435 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:39.948568 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:39.948790 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:39.949930 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:39.950894 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:39.951450 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:39.951554 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:39.952168 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:39.952173 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:39.952173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:39.952346 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:39.952378 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:39.952391 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:39.952472 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:39.952547 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 23:18:39.952571 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":811,"thread_start_us":74,"threads_started":1}
I20260809 23:18:39.956992 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:39.962390 7264 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:39.963049 7264 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:39.963231 7257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:39.958301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54167 (local address 127.2.76.254:38777)
0809 23:18:39.958329 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:39.958330 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:39.960241 (+ 1911us) server_negotiation.cc:407] Connection header received
0809 23:18:39.960327 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:39.960328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:39.960338 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:39.960389 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:39.960532 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:39.961147 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:39.962122 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:39.962349 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:39.962411 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:39.962965 (+ 554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:39.962966 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:39.962967 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:39.962971 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:39.962982 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:39.962986 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:39.963050 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:39.963165 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 23:18:39.963177 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 23:18:39.972949 7264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2121316cd78455cba5ac959e72aa312: Generated new TSK 0
I20260809 23:18:39.973088 7264 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:39.977056 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 7132
W20260809 23:18:39.988631 7097 connection.cc:535] client connection to 127.2.76.254:38777 recv error: Network error: failed to read from TLS socket (remote: 127.2.76.254:38777): Connection reset by peer (error 104)
2026-08-09T23:18:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1373 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:40 dist-test-slave-7vjv krb5kdc[7296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:40 dist-test-slave-7vjv krb5kdc[7296](info): set up 2 sockets
Aug 09 23:18:40 dist-test-slave-7vjv krb5kdc[7296](info): commencing operation
krb5kdc: starting...
W20260809 23:18:41.101985 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:41 dist-test-slave-7vjv krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:18:41.208422 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:43543
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:33559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:43543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:41.279922 7322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:41.280115 7322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:41.280179 7322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:41.281250 7322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:41.281287 7322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:41.281301 7322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:41.281344 7322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:41.281360 7322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:41.282701 7322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:33559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:43543
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:43543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:41.282923 7322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:41.283167 7322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:41.287536 7341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:41.287626 7339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:41.287568 7337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:41.287931 7322 server_base.cc:1034] running on GCE node
I20260809 23:18:41.288270 7322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:41.288582 7322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:41.289721 7322 hybrid_clock.cc:648] HybridClock initialized: now 1786317521289705 us; error 36 us; skew 500 ppm
Aug 09 23:18:41 dist-test-slave-7vjv krb5kdc[7296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317521, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:41.295157 7322 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:18:41.295534 7322 webserver.cc:460] Webserver started at http://127.2.76.254:42231/ using document root <none> and password file <none>
I20260809 23:18:41.295727 7322 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:41.295786 7322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:41.295918 7322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:41.296849 7322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "a147202c69b34565ab264eaa1a8a5a5d"
format_stamp: "Formatted at 2026-08-09 23:18:41 on dist-test-slave-7vjv"
server_key: "30ceee493220b96a405004b757d84ba9"
server_key_iv: "6970ab076d008e2e8ca5ca8aa13f6d0c"
server_key_version: "encryptionkey@0"
I20260809 23:18:41.297183 7322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "a147202c69b34565ab264eaa1a8a5a5d"
format_stamp: "Formatted at 2026-08-09 23:18:41 on dist-test-slave-7vjv"
server_key: "30ceee493220b96a405004b757d84ba9"
server_key_iv: "6970ab076d008e2e8ca5ca8aa13f6d0c"
server_key_version: "encryptionkey@0"
I20260809 23:18:41.298336 7322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:18:41.298959 7352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:41.299122 7322 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:41.299213 7322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "a147202c69b34565ab264eaa1a8a5a5d"
format_stamp: "Formatted at 2026-08-09 23:18:41 on dist-test-slave-7vjv"
server_key: "30ceee493220b96a405004b757d84ba9"
server_key_iv: "6970ab076d008e2e8ca5ca8aa13f6d0c"
server_key_version: "encryptionkey@0"
I20260809 23:18:41.299289 7322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:41.302170 7322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:41.303649 7322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:41.303795 7322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:41.307528 7322 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:43543
I20260809 23:18:41.307554 7433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:43543 every 8 connection(s)
I20260809 23:18:41.307822 7322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:41.308714 7434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:41.310513 7434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d: Bootstrap starting.
I20260809 23:18:41.310952 7434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:41.311197 7434 log.cc:824] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:41.311796 7434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d: No bootstrap required, opened a new log
I20260809 23:18:41.312278 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 7322
I20260809 23:18:41.312381 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:41.312569 2355 external_mini_cluster.cc:1428] Setting key 1ae4c463180a93406a7a2e9d7df26183
I20260809 23:18:41.313704 7434 raft_consensus.cc:348] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a147202c69b34565ab264eaa1a8a5a5d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43543 } }
I20260809 23:18:41.313853 7434 raft_consensus.cc:374] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:41.313889 7434 raft_consensus.cc:729] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a147202c69b34565ab264eaa1a8a5a5d, State: Initialized, Role: FOLLOWER
I20260809 23:18:41.314059 7434 consensus_queue.cc:260] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a147202c69b34565ab264eaa1a8a5a5d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43543 } }
I20260809 23:18:41.314154 7434 raft_consensus.cc:388] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:41.314229 7434 raft_consensus.cc:482] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:41.314287 7434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:41.314810 7441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:41.313010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41497 (local address 127.2.76.254:43543)
0809 23:18:41.313176 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:41.313183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:41.313212 (+ 29us) server_negotiation.cc:407] Connection header received
0809 23:18:41.313685 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:41.313689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:41.313711 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:41.313805 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:41.314380 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:41.314383 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:41.314383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:41.314387 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:41.314430 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:41.314440 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:41.314489 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:41.314589 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 23:18:41.314633 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260809 23:18:41.315204 7434 raft_consensus.cc:504] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a147202c69b34565ab264eaa1a8a5a5d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43543 } }
I20260809 23:18:41.315307 7434 leader_election.cc:302] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a147202c69b34565ab264eaa1a8a5a5d; no voters:
I20260809 23:18:41.315445 7434 leader_election.cc:288] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:41.315465 7442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:41.315565 7442 raft_consensus.cc:686] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [term 1 LEADER]: Becoming Leader. State: Replica: a147202c69b34565ab264eaa1a8a5a5d, State: Running, Role: LEADER
I20260809 23:18:41.315636 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:41.315666 7434 sys_catalog.cc:564] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:41.315694 7442 consensus_queue.cc:237] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a147202c69b34565ab264eaa1a8a5a5d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43543 } }
I20260809 23:18:41.316030 7442 sys_catalog.cc:455] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a147202c69b34565ab264eaa1a8a5a5d. Latest consensus state: current_term: 1 leader_uuid: "a147202c69b34565ab264eaa1a8a5a5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a147202c69b34565ab264eaa1a8a5a5d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43543 } } }
I20260809 23:18:41.316138 7442 sys_catalog.cc:458] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:41.316042 7443 sys_catalog.cc:455] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a147202c69b34565ab264eaa1a8a5a5d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a147202c69b34565ab264eaa1a8a5a5d" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43543 } } }
I20260809 23:18:41.316259 7443 sys_catalog.cc:458] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:41.316414 7454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:41.316902 7454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:41.318460 7441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:41.317286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37993 (local address 127.2.76.254:43543)
0809 23:18:41.317324 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:41.317326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:41.317447 (+ 121us) server_negotiation.cc:407] Connection header received
0809 23:18:41.317504 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:41.317504 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:41.317508 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:41.317570 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:41.318242 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:41.318243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:41.318243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:41.318257 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:41.318266 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:41.318269 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:41.318330 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:41.318383 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 23:18:41.318396 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:18:41.318701 7454 catalog_manager.cc:1347] Generated new cluster ID: 107384ea1ecd41c8b866655c9b530cd0
I20260809 23:18:41.318744 7454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:41.332028 7454 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:41.332180 7454 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:41.336216 7454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a147202c69b34565ab264eaa1a8a5a5d: Generated new TSK 0
I20260809 23:18:41.336369 7454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:41.343639 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 7322
2026-08-09T23:18:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1368 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:41 dist-test-slave-7vjv krb5kdc[7489](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:41 dist-test-slave-7vjv krb5kdc[7489](info): set up 2 sockets
Aug 09 23:18:41 dist-test-slave-7vjv krb5kdc[7489](info): commencing operation
krb5kdc: starting...
W20260809 23:18:42.477236 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:42 dist-test-slave-7vjv krb5kdc[7489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:18:42.584950 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:39585
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:36387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:39585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:42.659461 7516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:42.659693 7516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:42.659730 7516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:42.660849 7516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:42.660909 7516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:42.660931 7516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:42.660949 7516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:42.660977 7516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:42.662393 7516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:36387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:39585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:39585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:42.662674 7516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:42.662897 7516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:42.667435 7530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:42.667451 7529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:42.667586 7516 server_base.cc:1034] running on GCE node
W20260809 23:18:42.667454 7535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:42.667820 7516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:42.668133 7516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:42.669278 7516 hybrid_clock.cc:648] HybridClock initialized: now 1786317522669271 us; error 30 us; skew 500 ppm
Aug 09 23:18:42 dist-test-slave-7vjv krb5kdc[7489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317522, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:42.672413 7516 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:18:42.672995 7516 webserver.cc:460] Webserver started at http://127.2.76.254:46101/ using document root <none> and password file <none>
I20260809 23:18:42.673259 7516 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:42.673322 7516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:42.673499 7516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:42.674531 7516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "ab2e72d316c64e4baf37b7fc43a954f0"
format_stamp: "Formatted at 2026-08-09 23:18:42 on dist-test-slave-7vjv"
server_key: "3555673e0e341f9f58143f0e683d8645"
server_key_iv: "832759f13700011b8eaa35bde68efc1c"
server_key_version: "encryptionkey@0"
I20260809 23:18:42.674875 7516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "ab2e72d316c64e4baf37b7fc43a954f0"
format_stamp: "Formatted at 2026-08-09 23:18:42 on dist-test-slave-7vjv"
server_key: "3555673e0e341f9f58143f0e683d8645"
server_key_iv: "832759f13700011b8eaa35bde68efc1c"
server_key_version: "encryptionkey@0"
I20260809 23:18:42.676556 7516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:18:42.677595 7549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:42.677816 7516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:18:42.677911 7516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "ab2e72d316c64e4baf37b7fc43a954f0"
format_stamp: "Formatted at 2026-08-09 23:18:42 on dist-test-slave-7vjv"
server_key: "3555673e0e341f9f58143f0e683d8645"
server_key_iv: "832759f13700011b8eaa35bde68efc1c"
server_key_version: "encryptionkey@0"
I20260809 23:18:42.677983 7516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:42.700040 7516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:42.702566 7516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:42.702723 7516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:42.711031 7516 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:39585
I20260809 23:18:42.711059 7632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:39585 every 8 connection(s)
I20260809 23:18:42.711529 7516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:42.712636 7633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:42.716653 7633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0: Bootstrap starting.
I20260809 23:18:42.717192 7633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:42.717451 7633 log.cc:824] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:42.718165 7633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0: No bootstrap required, opened a new log
I20260809 23:18:42.719636 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 7516
I20260809 23:18:42.719734 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:42.719715 7633 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2e72d316c64e4baf37b7fc43a954f0" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39585 } }
I20260809 23:18:42.719897 7633 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:42.719901 2355 external_mini_cluster.cc:1428] Setting key 1f7f4d14241e35b5723e15244217ac6f
I20260809 23:18:42.719939 7633 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab2e72d316c64e4baf37b7fc43a954f0, State: Initialized, Role: FOLLOWER
I20260809 23:18:42.720088 7633 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2e72d316c64e4baf37b7fc43a954f0" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39585 } }
I20260809 23:18:42.720225 7633 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:42.720255 7633 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:42.720297 7633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:42.721607 7633 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2e72d316c64e4baf37b7fc43a954f0" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39585 } }
I20260809 23:18:42.721743 7633 leader_election.cc:302] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab2e72d316c64e4baf37b7fc43a954f0; no voters:
I20260809 23:18:42.721915 7633 leader_election.cc:288] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:42.721963 7641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:42.722262 7641 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [term 1 LEADER]: Becoming Leader. State: Replica: ab2e72d316c64e4baf37b7fc43a954f0, State: Running, Role: LEADER
I20260809 23:18:42.722343 7641 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2e72d316c64e4baf37b7fc43a954f0" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39585 } }
I20260809 23:18:42.722348 7633 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:42.722641 7641 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab2e72d316c64e4baf37b7fc43a954f0. Latest consensus state: current_term: 1 leader_uuid: "ab2e72d316c64e4baf37b7fc43a954f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2e72d316c64e4baf37b7fc43a954f0" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39585 } } }
I20260809 23:18:42.722703 7641 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:42.722651 7645 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab2e72d316c64e4baf37b7fc43a954f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2e72d316c64e4baf37b7fc43a954f0" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39585 } } }
I20260809 23:18:42.722743 7645 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0 [sys.catalog]: This master's current role is: LEADER
W20260809 23:18:42.724234 7670 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:18:42.724324 7670 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:18:42.724406 7648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:42.724542 7648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:42.725980 7648 catalog_manager.cc:1347] Generated new cluster ID: 77ac9aa187144e1c96279a9ad8f95ef2
I20260809 23:18:42.726027 7648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:18:42 dist-test-slave-7vjv krb5kdc[7489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:42.729732 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:42.720364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38475 (local address 127.2.76.254:39585)
0809 23:18:42.720503 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:42.720511 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:42.720545 (+ 34us) server_negotiation.cc:407] Connection header received
0809 23:18:42.720695 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:42.720702 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:42.720769 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:42.720911 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:42.721711 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:42.722908 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:42.723899 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:42.724368 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:42.724470 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:42.726737 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:42.726741 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:42.726741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:42.726751 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:42.728357 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:42.728687 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:42.728690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:42.728690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:42.728799 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:42.729008 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:42.729009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:42.729009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:42.729226 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:42.729390 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:42.729454 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 23:18:42.729587 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":67,"threads_started":1}
I20260809 23:18:42.730546 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:42.733121 7648 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:42.733683 7648 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:42.737535 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:42.731779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43705 (local address 127.2.76.254:39585)
0809 23:18:42.731832 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:42.731834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:42.732238 (+ 404us) server_negotiation.cc:407] Connection header received
0809 23:18:42.732303 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:42.732304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:42.732335 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:42.732379 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:42.732816 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:42.733365 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:42.734002 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:42.734189 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:42.734253 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:42.735240 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:42.735241 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:42.735241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:42.735247 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:42.736503 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:42.736821 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:42.736822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:42.736822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:42.736885 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:42.737069 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:42.737070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:42.737070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:42.737189 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:42.737272 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:42.737414 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 23:18:42.737488 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:18:42.737525 7648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab2e72d316c64e4baf37b7fc43a954f0: Generated new TSK 0
I20260809 23:18:42.737736 7648 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:42.739522 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 7516
2026-08-09T23:18:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:42 dist-test-slave-7vjv krb5kdc[7686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:42 dist-test-slave-7vjv krb5kdc[7686](info): set up 2 sockets
Aug 09 23:18:42 dist-test-slave-7vjv krb5kdc[7686](info): commencing operation
krb5kdc: starting...
W20260809 23:18:43.889381 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:43 dist-test-slave-7vjv krb5kdc[7686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:18:43.994263 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:38915
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:43119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:38915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:44.066871 7703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:44.067047 7703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:44.067068 7703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:44.068235 7703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:44.068274 7703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:44.068286 7703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:44.068296 7703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:44.068334 7703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:44.069594 7703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:43119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:38915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:38915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:44.069790 7703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:44.069967 7703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:44.074321 7717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:44.074332 7715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:44.074513 7703 server_base.cc:1034] running on GCE node
W20260809 23:18:44.074360 7714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:44.074795 7703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:44.075110 7703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:44.076259 7703 hybrid_clock.cc:648] HybridClock initialized: now 1786317524076254 us; error 35 us; skew 500 ppm
Aug 09 23:18:44 dist-test-slave-7vjv krb5kdc[7686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317524, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:44.078445 7703 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:18:44.078890 7703 webserver.cc:460] Webserver started at http://127.2.76.254:41881/ using document root <none> and password file <none>
I20260809 23:18:44.079113 7703 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:44.079175 7703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:44.079298 7703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:44.080276 7703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "0fa04aea82d7462ea0b0e741509c8a02"
format_stamp: "Formatted at 2026-08-09 23:18:44 on dist-test-slave-7vjv"
server_key: "6aea9305885a76a6828c55b498f53d1a"
server_key_iv: "a56547c397326b25580274f708785d0e"
server_key_version: "encryptionkey@0"
I20260809 23:18:44.080610 7703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "0fa04aea82d7462ea0b0e741509c8a02"
format_stamp: "Formatted at 2026-08-09 23:18:44 on dist-test-slave-7vjv"
server_key: "6aea9305885a76a6828c55b498f53d1a"
server_key_iv: "a56547c397326b25580274f708785d0e"
server_key_version: "encryptionkey@0"
I20260809 23:18:44.081830 7703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:18:44.082481 7729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:44.082674 7703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:44.082767 7703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "0fa04aea82d7462ea0b0e741509c8a02"
format_stamp: "Formatted at 2026-08-09 23:18:44 on dist-test-slave-7vjv"
server_key: "6aea9305885a76a6828c55b498f53d1a"
server_key_iv: "a56547c397326b25580274f708785d0e"
server_key_version: "encryptionkey@0"
I20260809 23:18:44.082854 7703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:44.110160 7703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:44.111683 7703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:44.111836 7703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:44.115607 7703 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:38915
I20260809 23:18:44.115640 7827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:38915 every 8 connection(s)
I20260809 23:18:44.115952 7703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:44.116752 7828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:44.118520 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 7703
I20260809 23:18:44.118638 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:44.118814 2355 external_mini_cluster.cc:1428] Setting key 40c0b92fa2705c8ca8a67f9eb2df1730
I20260809 23:18:44.119004 7828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02: Bootstrap starting.
I20260809 23:18:44.119645 7828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:44.119928 7828 log.cc:824] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:44.120601 7828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02: No bootstrap required, opened a new log
I20260809 23:18:44.122681 7828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa04aea82d7462ea0b0e741509c8a02" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38915 } }
I20260809 23:18:44.122838 7828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:44.122880 7828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa04aea82d7462ea0b0e741509c8a02, State: Initialized, Role: FOLLOWER
I20260809 23:18:44.123044 7828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa04aea82d7462ea0b0e741509c8a02" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38915 } }
I20260809 23:18:44.123145 7828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:44.123179 7828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:44.123229 7828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:44.123754 7828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa04aea82d7462ea0b0e741509c8a02" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38915 } }
I20260809 23:18:44.123871 7828 leader_election.cc:302] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fa04aea82d7462ea0b0e741509c8a02; no voters:
I20260809 23:18:44.123900 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:44.119249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34157 (local address 127.2.76.254:38915)
0809 23:18:44.119415 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:44.119423 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:44.119482 (+ 59us) server_negotiation.cc:407] Connection header received
0809 23:18:44.119903 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:44.119909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:44.119948 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:44.120083 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:44.120308 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:44.121140 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:44.122170 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:44.122491 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:44.122562 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:44.123317 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:44.123321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:44.123321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:44.123467 (+ 146us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:44.123491 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:44.123500 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:44.123549 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:44.123659 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 23:18:44.123701 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260809 23:18:44.124065 7828 leader_election.cc:288] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:44.124199 7838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:44.124297 7828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:44.124336 7838 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [term 1 LEADER]: Becoming Leader. State: Replica: 0fa04aea82d7462ea0b0e741509c8a02, State: Running, Role: LEADER
I20260809 23:18:44.124415 7838 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa04aea82d7462ea0b0e741509c8a02" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38915 } }
I20260809 23:18:44.124763 7841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fa04aea82d7462ea0b0e741509c8a02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa04aea82d7462ea0b0e741509c8a02" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38915 } } }
I20260809 23:18:44.124857 7841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:44.124884 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:44.124758 7844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fa04aea82d7462ea0b0e741509c8a02. Latest consensus state: current_term: 1 leader_uuid: "0fa04aea82d7462ea0b0e741509c8a02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa04aea82d7462ea0b0e741509c8a02" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 38915 } } }
I20260809 23:18:44.125025 7844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:44.125136 7851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:44.125587 7851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:44.127254 7851 catalog_manager.cc:1347] Generated new cluster ID: c1cf09fdf732437fb96b95207ae1ca81
I20260809 23:18:44.127303 7851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:44.129480 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:44.126494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58509 (local address 127.2.76.254:38915)
0809 23:18:44.126526 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:44.126528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:44.126744 (+ 216us) server_negotiation.cc:407] Connection header received
0809 23:18:44.126787 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:44.126787 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:44.126796 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:44.126845 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:44.127011 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:44.127578 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:44.128471 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:44.128723 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:44.128770 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:44.129305 (+ 535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:44.129306 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:44.129306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:44.129309 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:44.129320 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:44.129324 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:44.129365 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:44.129406 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 23:18:44.129424 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:18:44.132368 7851 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:44.132788 7851 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:44.145182 7851 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fa04aea82d7462ea0b0e741509c8a02: Generated new TSK 0
I20260809 23:18:44.145344 7851 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:44.148183 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 7703
2026-08-09T23:18:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1411 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:44 dist-test-slave-7vjv krb5kdc[7879](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:44 dist-test-slave-7vjv krb5kdc[7879](info): set up 2 sockets
Aug 09 23:18:44 dist-test-slave-7vjv krb5kdc[7879](info): commencing operation
krb5kdc: starting...
W20260809 23:18:45.301976 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:45 dist-test-slave-7vjv krb5kdc[7879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:18:45.406008 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:36685
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:41243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:36685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:45.480769 7900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:45.480963 7900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:45.480984 7900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:45.481998 7900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:45.482031 7900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:45.482045 7900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:45.482057 7900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:45.482069 7900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:45.483325 7900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:41243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:36685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:36685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:45.483495 7900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:45.483645 7900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:45.488189 7907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:45.488197 7910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:45.488226 7900 server_base.cc:1034] running on GCE node
W20260809 23:18:45.488219 7913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:45.488678 7900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:45.488945 7900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:45.490116 7900 hybrid_clock.cc:648] HybridClock initialized: now 1786317525490104 us; error 41 us; skew 500 ppm
Aug 09 23:18:45 dist-test-slave-7vjv krb5kdc[7879](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317525, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:45.492331 7900 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:18:45.492774 7900 webserver.cc:460] Webserver started at http://127.2.76.254:38181/ using document root <none> and password file <none>
I20260809 23:18:45.492990 7900 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:45.493054 7900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:45.493192 7900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:45.494094 7900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "f1998b4c79de4b46b857648968261ad3"
format_stamp: "Formatted at 2026-08-09 23:18:45 on dist-test-slave-7vjv"
server_key: "95ec5930b142b82edb2d73feb28cc065"
server_key_iv: "6cfd65efc92d147f94c47a30e7e78330"
server_key_version: "encryptionkey@0"
I20260809 23:18:45.494485 7900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "f1998b4c79de4b46b857648968261ad3"
format_stamp: "Formatted at 2026-08-09 23:18:45 on dist-test-slave-7vjv"
server_key: "95ec5930b142b82edb2d73feb28cc065"
server_key_iv: "6cfd65efc92d147f94c47a30e7e78330"
server_key_version: "encryptionkey@0"
I20260809 23:18:45.495684 7900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:18:45.496447 7926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:45.496614 7900 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:18:45.496718 7900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "f1998b4c79de4b46b857648968261ad3"
format_stamp: "Formatted at 2026-08-09 23:18:45 on dist-test-slave-7vjv"
server_key: "95ec5930b142b82edb2d73feb28cc065"
server_key_iv: "6cfd65efc92d147f94c47a30e7e78330"
server_key_version: "encryptionkey@0"
I20260809 23:18:45.496790 7900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:45.499665 7900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:45.501279 7900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:45.501456 7900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:45.505328 7900 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:36685
I20260809 23:18:45.505358 8027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:36685 every 8 connection(s)
I20260809 23:18:45.505640 7900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:45.506196 8028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:45.508015 8028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3: Bootstrap starting.
I20260809 23:18:45.508550 8028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:45.508759 8028 log.cc:824] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:45.509287 8028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3: No bootstrap required, opened a new log
I20260809 23:18:45.509733 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 7900
I20260809 23:18:45.509837 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:45.510012 2355 external_mini_cluster.cc:1428] Setting key bfc6731a9b689204f10759d498a6ea4f
I20260809 23:18:45.511901 8028 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1998b4c79de4b46b857648968261ad3" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36685 } }
I20260809 23:18:45.512054 8028 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:18:45.512110 8028 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1998b4c79de4b46b857648968261ad3, State: Initialized, Role: FOLLOWER
I20260809 23:18:45.512292 8028 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1998b4c79de4b46b857648968261ad3" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36685 } }
I20260809 23:18:45.512406 8028 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:18:45.512449 8028 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:18:45.512496 8028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:45.513033 8033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:45.510522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35389 (local address 127.2.76.254:36685)
0809 23:18:45.510727 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:45.510735 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:45.510770 (+ 35us) server_negotiation.cc:407] Connection header received
0809 23:18:45.511859 (+ 1089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:45.511867 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:45.511899 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:45.512044 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:45.512642 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:45.512646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:45.512646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:45.512650 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:45.512687 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:45.512699 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:45.512766 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:45.512815 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 23:18:45.512850 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":83,"threads_started":1}
I20260809 23:18:45.513127 8028 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1998b4c79de4b46b857648968261ad3" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36685 } }
I20260809 23:18:45.513209 8028 leader_election.cc:302] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1998b4c79de4b46b857648968261ad3; no voters:
I20260809 23:18:45.513373 8028 leader_election.cc:288] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:18:45.513401 8034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:45.513490 8034 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [term 1 LEADER]: Becoming Leader. State: Replica: f1998b4c79de4b46b857648968261ad3, State: Running, Role: LEADER
I20260809 23:18:45.513599 8028 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:45.513608 8034 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1998b4c79de4b46b857648968261ad3" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36685 } }
I20260809 23:18:45.514140 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:45.514091 8037 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1998b4c79de4b46b857648968261ad3. Latest consensus state: current_term: 1 leader_uuid: "f1998b4c79de4b46b857648968261ad3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1998b4c79de4b46b857648968261ad3" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36685 } } }
I20260809 23:18:45.514179 8037 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:45.514319 8036 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1998b4c79de4b46b857648968261ad3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1998b4c79de4b46b857648968261ad3" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 36685 } } }
I20260809 23:18:45.514509 8036 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:45.514674 8049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:45.515129 8049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:45.516736 8049 catalog_manager.cc:1347] Generated new cluster ID: c5ffdcffc6ea4953b8182064a604047b
I20260809 23:18:45.516788 8049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:45.517433 8033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:45.515754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44689 (local address 127.2.76.254:36685)
0809 23:18:45.515785 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:45.515787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:45.516383 (+ 596us) server_negotiation.cc:407] Connection header received
0809 23:18:45.516493 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:45.516494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:45.516497 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:45.516569 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:45.517213 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:45.517214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:45.517214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:18:45.517219 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:45.517231 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:18:45.517235 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:45.517281 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:45.517333 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 23:18:45.517347 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 23:18:45.539918 8049 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:45.540086 8049 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:45.543004 8049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1998b4c79de4b46b857648968261ad3: Generated new TSK 0
I20260809 23:18:45.543174 8049 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:18:45.550783 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 7900
2026-08-09T23:18:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1394 ms)
[----------] 18 tests from AuthTokenIssuingTest (25122 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:45 dist-test-slave-7vjv krb5kdc[8078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:45 dist-test-slave-7vjv krb5kdc[8078](info): set up 2 sockets
Aug 09 23:18:45 dist-test-slave-7vjv krb5kdc[8078](info): commencing operation
krb5kdc: starting...
W20260809 23:18:46.716398 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:46 dist-test-slave-7vjv krb5kdc[8078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:18:46.821990 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:43285
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:33771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:43285,127.2.76.253:38687,127.2.76.252:34517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:46.894667 8099 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:18:46.894888 8099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:46.894929 8099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:46.894989 8099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:46.896041 8099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:46.896102 8099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:46.896157 8099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:46.896183 8099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:46.896209 8099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:46.897552 8099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:33771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:43285,127.2.76.253:38687,127.2.76.252:34517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:43285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:46.897806 8099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:46.898007 8099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:46.902256 8110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:46.902287 8111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:46.902277 8113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:46.902545 8099 server_base.cc:1034] running on GCE node
I20260809 23:18:46.902756 8099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:46.903002 8099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:46.904156 8099 hybrid_clock.cc:648] HybridClock initialized: now 1786317526904127 us; error 41 us; skew 500 ppm
Aug 09 23:18:46 dist-test-slave-7vjv krb5kdc[8078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317526, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:46.909911 8099 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:18:46.910775 8099 webserver.cc:460] Webserver started at http://127.2.76.254:43653/ using document root <none> and password file <none>
I20260809 23:18:46.911000 8099 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:46.911042 8099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:46.911173 8099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:46.912180 8099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "b5013caacf814af79070dba4aec7a316"
format_stamp: "Formatted at 2026-08-09 23:18:46 on dist-test-slave-7vjv"
server_key: "4de08ffe2ff005a8b2906a5dd9261340"
server_key_iv: "ee85f37050f2a5505a5e42bcf0588e49"
server_key_version: "encryptionkey@0"
I20260809 23:18:46.912537 8099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "b5013caacf814af79070dba4aec7a316"
format_stamp: "Formatted at 2026-08-09 23:18:46 on dist-test-slave-7vjv"
server_key: "4de08ffe2ff005a8b2906a5dd9261340"
server_key_iv: "ee85f37050f2a5505a5e42bcf0588e49"
server_key_version: "encryptionkey@0"
I20260809 23:18:46.913674 8099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:18:46.914290 8122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:46.914456 8099 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:18:46.914547 8099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "b5013caacf814af79070dba4aec7a316"
format_stamp: "Formatted at 2026-08-09 23:18:46 on dist-test-slave-7vjv"
server_key: "4de08ffe2ff005a8b2906a5dd9261340"
server_key_iv: "ee85f37050f2a5505a5e42bcf0588e49"
server_key_version: "encryptionkey@0"
I20260809 23:18:46.914609 8099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:46.922340 8099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:46.923961 8099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:46.924146 8099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:46.927829 8099 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:43285
I20260809 23:18:46.927873 8222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:43285 every 8 connection(s)
I20260809 23:18:46.928225 8099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:46.929096 8223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:46.930394 8223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:18:46 dist-test-slave-7vjv krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317526, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:46.935606 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 8099
I20260809 23:18:46.935719 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:46.935925 2355 external_mini_cluster.cc:1428] Setting key 67caa5d405da2f8298ba4077f30c396a
I20260809 23:18:46.939008 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:46.930988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60284 (local address 127.2.76.254:43285)
0809 23:18:46.931139 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:46.931143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:46.931178 (+ 35us) server_negotiation.cc:407] Connection header received
0809 23:18:46.931653 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:46.931655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:46.931669 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:46.931740 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:46.932255 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:46.932967 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:46.933664 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:46.933917 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:46.933982 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:46.935694 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:46.935703 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:46.935704 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:46.935712 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:46.937536 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:46.937893 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:46.937896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:46.937896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:46.937973 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:46.938181 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:46.938181 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:46.938182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:46.938373 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:46.938460 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:46.938662 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 23:18:46.938844 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":110,"server-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260809 23:18:46.939014 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:46.930914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:43285 (local address 127.0.0.1:60284)
0809 23:18:46.931072 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:46.931108 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:46.931500 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:46.931767 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:46.931769 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:46.931971 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:46.932170 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:46.932176 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:46.933043 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:46.933044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:46.933599 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:46.933600 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:46.933998 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:46.934005 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:46.934050 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:46.934436 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:46.934443 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:46.935609 (+ 1166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:46.937624 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:46.937630 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:46.937631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:46.937795 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:46.938043 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:46.938043 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:46.938044 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:46.938117 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:46.938455 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:46.938460 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:46.938539 (+ 79us) client_negotiation.cc:769] Sending connection context
0809 23:18:46.938619 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 23:18:46.938795 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260809 23:18:46.939769 8223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:18:46.940068 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:46.939964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:38687 (local address 127.0.0.1:40672)
0809 23:18:46.940013 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:46.940049 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:38687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 23:18:46.940229 8123 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.76.253:38687: connect: Connection refused (error 111)
I20260809 23:18:46.940655 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:46.940559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:38687 (local address 127.0.0.1:40682)
0809 23:18:46.940610 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:46.940636 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:38687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 23:18:46.940781 8223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.253:38687: Network error: Client connection negotiation failed: client connection to 127.2.76.253:38687: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.2.76.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.253.keytab.
Entry for principal kudu/127.2.76.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.253.keytab.
I20260809 23:18:46.958432 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.253:38687
--webserver_interface=127.2.76.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.253.keytab
--principal=kudu/127.2.76.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:33771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:43285,127.2.76.253:38687,127.2.76.252:34517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:18:46.990042 8223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } attempt: 1
I20260809 23:18:46.990420 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:46.990312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:38687 (local address 127.0.0.1:40698)
0809 23:18:46.990366 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:46.990398 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:38687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 23:18:46.990718 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:46.990632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:38687 (local address 127.0.0.1:40712)
0809 23:18:46.990672 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:46.990694 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:38687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 23:18:46.990789 8223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.253:38687: Network error: Client connection negotiation failed: client connection to 127.2.76.253:38687: connect: Connection refused (error 111)
W20260809 23:18:47.033004 8236 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:18:47.033174 8236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:47.033188 8236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:47.033233 8236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:47.034214 8236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:47.034248 8236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:47.034260 8236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:47.034293 8236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:47.034317 8236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:47.035581 8236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:33771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:43285,127.2.76.253:38687,127.2.76.252:34517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.253.keytab
--principal=kudu/127.2.76.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.253:38687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.76.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:47.035749 8236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:47.035945 8236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:47.041018 8249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:47.041074 8236 server_base.cc:1034] running on GCE node
W20260809 23:18:47.041038 8251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:47.041103 8246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:47.041419 8236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:47.041718 8236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:47.042865 8236 hybrid_clock.cc:648] HybridClock initialized: now 1786317527042844 us; error 31 us; skew 500 ppm
Aug 09 23:18:47 dist-test-slave-7vjv krb5kdc[8078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317527, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:47.045123 8236 init.cc:377] Logged in from keytab as kudu/127.2.76.253@KRBTEST.COM (short username kudu)
I20260809 23:18:47.045492 8236 webserver.cc:460] Webserver started at http://127.2.76.253:35939/ using document root <none> and password file <none>
I20260809 23:18:47.045696 8236 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:47.045753 8236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:47.045881 8236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:47.046736 8236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/data/instance:
uuid: "c98607b3aa694cf7a560f6984fc8888b"
format_stamp: "Formatted at 2026-08-09 23:18:47 on dist-test-slave-7vjv"
server_key: "28588b5ba4c68574dc9481112df8d229"
server_key_iv: "69f7c4a62d07f3f3b02038090b9d75a8"
server_key_version: "encryptionkey@0"
I20260809 23:18:47.047120 8236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/wal/instance:
uuid: "c98607b3aa694cf7a560f6984fc8888b"
format_stamp: "Formatted at 2026-08-09 23:18:47 on dist-test-slave-7vjv"
server_key: "28588b5ba4c68574dc9481112df8d229"
server_key_iv: "69f7c4a62d07f3f3b02038090b9d75a8"
server_key_version: "encryptionkey@0"
I20260809 23:18:47.048450 8236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:18:47.049072 8260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:47.049214 8236 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:47.049321 8236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/wal
uuid: "c98607b3aa694cf7a560f6984fc8888b"
format_stamp: "Formatted at 2026-08-09 23:18:47 on dist-test-slave-7vjv"
server_key: "28588b5ba4c68574dc9481112df8d229"
server_key_iv: "69f7c4a62d07f3f3b02038090b9d75a8"
server_key_version: "encryptionkey@0"
I20260809 23:18:47.049391 8236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:47.058990 8223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } attempt: 2
I20260809 23:18:47.059315 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.059213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:38687 (local address 127.0.0.1:40724)
0809 23:18:47.059264 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.059294 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:38687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 23:18:47.059623 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.059527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:38687 (local address 127.0.0.1:40740)
0809 23:18:47.059576 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.059601 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:38687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 23:18:47.059695 8223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.253:38687: Network error: Client connection negotiation failed: client connection to 127.2.76.253:38687: connect: Connection refused (error 111)
I20260809 23:18:47.061843 8236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:47.063499 8236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:47.063638 8236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:47.067288 8236 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.253:38687
I20260809 23:18:47.067315 8353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.253:38687 every 8 connection(s)
I20260809 23:18:47.067606 8236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/data/info.pb
I20260809 23:18:47.068495 8354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:47.069890 8354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:18:47.072410 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 8236
I20260809 23:18:47.072531 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-1/wal/instance
I20260809 23:18:47.072741 2355 external_mini_cluster.cc:1428] Setting key 0272a1718eecaf5ef6beab3b07d2f803
Aug 09 23:18:47 dist-test-slave-7vjv krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317527, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.253@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:47.078981 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.070425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60292 (local address 127.2.76.254:43285)
0809 23:18:47.070488 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.070490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.070651 (+ 161us) server_negotiation.cc:407] Connection header received
0809 23:18:47.071042 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.071043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.071054 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.071100 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.071664 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.072160 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.073217 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.073547 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.073636 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.076200 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:47.076201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:47.076201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:47.076207 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:47.077613 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.078100 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.078101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.078101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.078149 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.078415 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.078416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.078416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.078510 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:47.078556 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:47.078872 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 23:18:47.078930 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:18:47.079321 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.070392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:43285 (local address 127.0.0.1:60292)
0809 23:18:47.070550 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.070581 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:47.070962 (+ 381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:47.071144 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:47.071147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:47.071385 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:47.071566 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.071570 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.072234 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.072235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.073134 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.073136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.073655 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.073657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.073781 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.074221 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:47.074229 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:47.076047 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:47.077709 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.077717 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.077719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.078022 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.078210 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.078211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.078212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.078348 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.078574 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:47.078581 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:47.078737 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 23:18:47.078855 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 23:18:47.079043 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":86,"threads_started":1}
I20260809 23:18:47.079923 8354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.76.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.252@KRBTEST.COM" created.
Aug 09 23:18:47 dist-test-slave-7vjv krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317527, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.253@KRBTEST.COM for kudu/127.2.76.253@KRBTEST.COM
I20260809 23:18:47.090976 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.080204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:38687 (local address 127.0.0.1:40754)
0809 23:18:47.080258 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.080266 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:47.080307 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:47.080554 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:47.080555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:47.080713 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:47.080779 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.080780 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.081502 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.081503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.082286 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.082289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.082695 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.082696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.082743 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.083016 (+ 273us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:47.083019 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:47.085490 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:47.090019 (+ 4529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.090020 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.090021 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.090185 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.090398 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.090399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.090400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.090447 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.090693 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:47.090694 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:47.090725 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 23:18:47.090786 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 23:18:47.090868 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 23:18:47.091058 8362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.080274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40754 (local address 127.2.76.253:38687)
0809 23:18:47.080402 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.080418 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.080430 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:18:47.080443 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.080444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.080471 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.080524 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.080855 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.081414 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.082362 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.082633 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.082681 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.085589 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:47.085595 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:47.085596 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:47.085601 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:47.089921 (+ 4320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.090271 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.090273 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.090274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.090326 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.090514 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.090515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.090515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.090629 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:47.090709 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:47.090812 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 23:18:47.090927 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":109,"thread_start_us":88,"threads_started":1}
I20260809 23:18:47.091600 8354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:18:47.091853 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.091764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:34517 (local address 127.0.0.1:54586)
0809 23:18:47.091801 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.091838 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:34517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 23:18:47.091979 8268 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.76.252:34517: connect: Connection refused (error 111)
I20260809 23:18:47.092401 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.092317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:34517 (local address 127.0.0.1:54600)
0809 23:18:47.092361 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.092382 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:34517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 23:18:47.092526 8354 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.252:34517: Network error: Client connection negotiation failed: client connection to 127.2.76.252:34517: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.252.keytab.
Entry for principal kudu/127.2.76.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.252.keytab.
I20260809 23:18:47.098232 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.252:34517
--webserver_interface=127.2.76.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.252.keytab
--principal=kudu/127.2.76.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:33771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:43285,127.2.76.253:38687,127.2.76.252:34517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:18:47.141746 8354 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } attempt: 1
I20260809 23:18:47.142119 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.142013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:34517 (local address 127.0.0.1:54606)
0809 23:18:47.142064 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.142096 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:34517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 23:18:47.142403 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.142318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:34517 (local address 127.0.0.1:54618)
0809 23:18:47.142358 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.142380 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:34517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 23:18:47.142483 8354 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.252:34517: Network error: Client connection negotiation failed: client connection to 127.2.76.252:34517: connect: Connection refused (error 111)
I20260809 23:18:47.150928 8223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } attempt: 3
Aug 09 23:18:47 dist-test-slave-7vjv krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317526, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for kudu/127.2.76.253@KRBTEST.COM
I20260809 23:18:47.157174 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.151171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:38687 (local address 127.0.0.1:40760)
0809 23:18:47.151220 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.151229 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:47.151285 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:47.151409 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:47.151410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:47.151612 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:47.151693 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.151695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.152371 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.152372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.152956 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.152959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.153331 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.153331 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.153407 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.153842 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:47.153846 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:47.154872 (+ 1026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:47.156292 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.156293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.156293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.156442 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.156632 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.156633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.156633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.156685 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.156903 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:47.156904 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:47.156942 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 23:18:47.157010 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 23:18:47.157095 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 23:18:47.157172 8362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.151247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40760 (local address 127.2.76.253:38687)
0809 23:18:47.151302 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.151304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.151313 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:18:47.151335 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.151336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.151345 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.151405 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.151755 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.152294 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.153059 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.153264 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.153331 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.154953 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:47.154955 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:47.154955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:47.154966 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:47.156223 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.156514 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.156516 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.156516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.156568 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.156750 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.156751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.156751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.156845 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:47.156910 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:47.157026 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 23:18:47.157095 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 23:18:47.157568 8223 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:18:47.157871 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.157785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:34517 (local address 127.0.0.1:54622)
0809 23:18:47.157828 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.157853 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:34517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 23:18:47.158128 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.158050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:34517 (local address 127.0.0.1:54634)
0809 23:18:47.158090 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.158111 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:34517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 23:18:47.158210 8223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.252:34517: Network error: Client connection negotiation failed: client connection to 127.2.76.252:34517: connect: Connection refused (error 111)
W20260809 23:18:47.170830 8368 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:18:47.171032 8368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:47.171048 8368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:47.171067 8368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:47.172142 8368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:47.172188 8368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:47.172202 8368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:47.172230 8368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:47.172256 8368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:47.173597 8368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:33771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:43285,127.2.76.253:38687,127.2.76.252:34517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.252.keytab
--principal=kudu/127.2.76.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.252:34517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.76.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:47.173825 8368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:47.174023 8368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:18:47.178453 8368 server_base.cc:1034] running on GCE node
W20260809 23:18:47.178397 8374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:47.178416 8376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:47.178372 8373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:47.178912 8368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:47.179178 8368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:47.180330 8368 hybrid_clock.cc:648] HybridClock initialized: now 1786317527180321 us; error 24 us; skew 500 ppm
Aug 09 23:18:47 dist-test-slave-7vjv krb5kdc[8078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317527, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:47.182466 8368 init.cc:377] Logged in from keytab as kudu/127.2.76.252@KRBTEST.COM (short username kudu)
I20260809 23:18:47.182843 8368 webserver.cc:460] Webserver started at http://127.2.76.252:42909/ using document root <none> and password file <none>
I20260809 23:18:47.183046 8368 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:47.183109 8368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:47.183246 8368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:47.184140 8368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/data/instance:
uuid: "6b5ae70642d9477bb7da77226258937e"
format_stamp: "Formatted at 2026-08-09 23:18:47 on dist-test-slave-7vjv"
server_key: "62e832bb270ebc6f7ceca3e3b2c4a222"
server_key_iv: "d3357b1dad09180dc0722358a7697a4f"
server_key_version: "encryptionkey@0"
I20260809 23:18:47.184468 8368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/wal/instance:
uuid: "6b5ae70642d9477bb7da77226258937e"
format_stamp: "Formatted at 2026-08-09 23:18:47 on dist-test-slave-7vjv"
server_key: "62e832bb270ebc6f7ceca3e3b2c4a222"
server_key_iv: "d3357b1dad09180dc0722358a7697a4f"
server_key_version: "encryptionkey@0"
I20260809 23:18:47.185653 8368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:18:47.186288 8385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:47.186439 8368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:18:47.186535 8368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/wal
uuid: "6b5ae70642d9477bb7da77226258937e"
format_stamp: "Formatted at 2026-08-09 23:18:47 on dist-test-slave-7vjv"
server_key: "62e832bb270ebc6f7ceca3e3b2c4a222"
server_key_iv: "d3357b1dad09180dc0722358a7697a4f"
server_key_version: "encryptionkey@0"
I20260809 23:18:47.186612 8368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:47.189342 8223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } attempt: 1
I20260809 23:18:47.189677 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.189565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:34517 (local address 127.0.0.1:54636)
0809 23:18:47.189618 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.189649 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:34517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 23:18:47.189940 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.189877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:34517 (local address 127.0.0.1:54650)
0809 23:18:47.189908 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.189927 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:34517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 23:18:47.190028 8223 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.252:34517: Network error: Client connection negotiation failed: client connection to 127.2.76.252:34517: connect: Connection refused (error 111)
I20260809 23:18:47.195657 8368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:47.197248 8368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:47.197351 8368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:47.200894 8368 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.252:34517
I20260809 23:18:47.200958 8481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.252:34517 every 8 connection(s)
I20260809 23:18:47.201217 8368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/data/info.pb
I20260809 23:18:47.202082 8482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:47.202219 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 8368
I20260809 23:18:47.202350 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786317238298104-2355-0/minicluster-data/master-2/wal/instance
I20260809 23:18:47.202538 2355 external_mini_cluster.cc:1428] Setting key 48c218910d24964556c689c998ee8808
I20260809 23:18:47.204208 8482 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:18:47 dist-test-slave-7vjv krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.252@KRBTEST.COM
Aug 09 23:18:47 dist-test-slave-7vjv krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317527, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.252@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:47.210608 8354 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } attempt: 2
I20260809 23:18:47.212348 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.202972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54660 (local address 127.2.76.252:34517)
0809 23:18:47.203145 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.203151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.203172 (+ 21us) server_negotiation.cc:407] Connection header received
0809 23:18:47.203528 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.203533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.203571 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.203668 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.204439 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.205567 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.206526 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.207051 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.207151 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.209291 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:47.209299 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:47.209299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:47.209306 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:47.210789 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.211149 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.211151 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.211152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.211247 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.211448 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.211450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.211450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.211709 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:47.211820 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:47.212022 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 23:18:47.212159 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260809 23:18:47.212352 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.204846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60308 (local address 127.2.76.254:43285)
0809 23:18:47.204908 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.204910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.205070 (+ 160us) server_negotiation.cc:407] Connection header received
0809 23:18:47.205131 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.205131 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.205139 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.205180 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.205647 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.206302 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.207280 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.207563 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.207629 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.209643 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:47.209645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:47.209645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:47.209651 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:47.211133 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.211515 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.211517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.211517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.211575 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.211792 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.211793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.211793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.211916 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:47.211975 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:47.212205 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 23:18:47.212276 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 23:18:47.212638 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.204810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:43285 (local address 127.0.0.1:60308)
0809 23:18:47.204966 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.205008 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:47.205080 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:47.205213 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:47.205216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:47.205506 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:47.205575 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.205576 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.206381 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.206382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.207190 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.207192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.207661 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.207662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.207718 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.208172 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:47.208184 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:47.209568 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:47.211177 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.211182 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.211183 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.211413 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.211643 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.211643 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.211644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.211699 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.212012 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:47.212019 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:47.212064 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 23:18:47.212469 (+ 405us) client_negotiation.cc:240] Negotiation successful
0809 23:18:47.212577 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260809 23:18:47.213197 8482 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:18:47 dist-test-slave-7vjv krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317527, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.253@KRBTEST.COM for kudu/127.2.76.252@KRBTEST.COM
Aug 09 23:18:47 dist-test-slave-7vjv krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317527, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.252@KRBTEST.COM for kudu/127.2.76.253@KRBTEST.COM
Aug 09 23:18:47 dist-test-slave-7vjv krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:47.217840 8356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.210847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:34517 (local address 127.0.0.1:54668)
0809 23:18:47.210892 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.210901 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:47.210937 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:47.211451 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:47.211453 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:47.211618 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:47.211663 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.211665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.212494 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.212496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.213184 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.213186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.213603 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.213605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.213659 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.213993 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:47.213997 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:47.215143 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:47.216847 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.216848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.216849 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.217042 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.217243 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.217244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.217245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.217306 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.217516 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:47.217517 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:47.217580 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 23:18:47.217653 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:18:47.217732 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 23:18:47.217909 8491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.211207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54668 (local address 127.2.76.252:34517)
0809 23:18:47.211328 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.211330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.211337 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:18:47.211353 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.211354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.211364 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.211440 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.211744 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.212385 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.213257 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.213524 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.213787 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.215216 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:47.215217 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:47.215217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:47.215223 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:47.216767 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.217117 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.217118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.217118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.217172 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.217362 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.217363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.217363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.217446 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:47.217490 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:47.217699 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 23:18:47.217768 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":81,"threads_started":1}
I20260809 23:18:47.219549 8354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b: Bootstrap starting.
I20260809 23:18:47.220518 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.213495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60310 (local address 127.2.76.254:43285)
0809 23:18:47.213523 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.213524 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.213596 (+ 72us) server_negotiation.cc:407] Connection header received
0809 23:18:47.213642 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.213643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.213651 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.213695 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.214197 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.214783 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.215553 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.215724 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.215785 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.218281 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:47.218282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:47.218282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:47.218288 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:47.219371 (+ 1083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.219759 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.219760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.219761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.219813 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.220015 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.220015 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.220016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.220170 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:47.220334 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:47.220367 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 23:18:47.220435 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 23:18:47.220535 8354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:47.220839 8354 log.cc:824] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:47.221652 8362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.213448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40776 (local address 127.2.76.253:38687)
0809 23:18:47.213483 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.213485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.213495 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:18:47.213555 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.213556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.213566 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.213610 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.213917 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.214407 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.214988 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.215250 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.215322 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.218332 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:47.218334 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:47.218334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:47.218340 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:47.220324 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.220792 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.220793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.220794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.220848 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.221054 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.221055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.221055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.221205 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:47.221266 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:47.221469 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 23:18:47.221562 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 23:18:47.221722 8354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b: No bootstrap required, opened a new log
I20260809 23:18:47.221916 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.213407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:38687 (local address 127.0.0.1:40776)
0809 23:18:47.213466 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.213475 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:47.213507 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:47.213650 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:47.213651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:47.213782 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:47.213835 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.213837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.214471 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.214471 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.214900 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.214902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.215324 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.215325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.215387 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.216319 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:47.216323 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:47.218247 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:47.220411 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.220412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.220412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.220709 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.220922 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.220922 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.220923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.220988 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.221293 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:47.221293 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:47.221328 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 23:18:47.221761 (+ 433us) client_negotiation.cc:240] Negotiation successful
0809 23:18:47.221845 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 23:18:47.222237 8482 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:18:47.223841 8354 raft_consensus.cc:348] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } }
I20260809 23:18:47.224010 8354 raft_consensus.cc:729] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c98607b3aa694cf7a560f6984fc8888b, State: Initialized, Role: FOLLOWER
I20260809 23:18:47.224149 8354 consensus_queue.cc:260] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } }
I20260809 23:18:47.224495 8354 sys_catalog.cc:564] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b [sys.catalog]: configured and running, proceeding with master startup.
W20260809 23:18:47.226305 8511 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:18:47.226394 8511 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 23:18:47 dist-test-slave-7vjv krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.253@KRBTEST.COM
Aug 09 23:18:47 dist-test-slave-7vjv krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317527, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.252@KRBTEST.COM for kudu/127.2.76.252@KRBTEST.COM
I20260809 23:18:47.226542 8236 master_runner.cc:186] Error getting master registration for 127.2.76.254:43285: OK, instance_id { permanent_uuid: "b5013caacf814af79070dba4aec7a316" instance_seqno: 1786317526926880 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:18:47.228601 8236 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:18:47.229774 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.222436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:34517 (local address 127.0.0.1:54678)
0809 23:18:47.223219 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.223230 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:47.223274 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:47.223395 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:47.223395 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:47.223541 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:47.223574 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.223576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.224451 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.224452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.225165 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.225168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.225656 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.225657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.225710 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.226028 (+ 318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:47.226032 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:47.227273 (+ 1241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:47.228879 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.228879 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.228880 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.229029 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.229211 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.229211 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.229212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.229258 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.229493 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:47.229493 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:47.229533 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 23:18:47.229606 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:18:47.229688 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767}
I20260809 23:18:47.229830 8491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.222477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54678 (local address 127.2.76.252:34517)
0809 23:18:47.222511 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.222512 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.223280 (+ 768us) server_negotiation.cc:407] Connection header received
0809 23:18:47.223324 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.223325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.223335 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.223393 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.223653 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.224378 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.225243 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.225579 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.225657 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.227349 (+ 1692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:47.227349 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:47.227350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:47.227354 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:47.228797 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.229102 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.229103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.229103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.229148 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.229321 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.229322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.229322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.229431 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:47.229498 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:47.229624 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 23:18:47.229735 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":14}
I20260809 23:18:47.230608 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.221276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40786 (local address 127.2.76.253:38687)
0809 23:18:47.221424 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.221425 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.221431 (+ 6us) server_negotiation.cc:407] Connection header received
0809 23:18:47.221446 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.221447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.221456 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.221547 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.222102 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.223112 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.223953 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.224310 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.224391 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.227740 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:47.227742 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:47.227743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:47.227749 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:47.229445 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.229794 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.229795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.229795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.229846 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.230038 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.230038 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.230039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.230140 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:47.230244 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:47.230432 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 23:18:47.230505 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260809 23:18:47.230738 8482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e: Bootstrap starting.
I20260809 23:18:47.231298 8482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:47.231482 8482 log.cc:824] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:47.232048 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:47.232084 8482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e: No bootstrap required, opened a new log
I20260809 23:18:47.233460 8482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } }
I20260809 23:18:47.233584 8482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b5ae70642d9477bb7da77226258937e, State: Initialized, Role: FOLLOWER
I20260809 23:18:47.233649 8482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } }
I20260809 23:18:47.232988 8178 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b5013caacf814af79070dba4aec7a316"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60310
I20260809 23:18:47.233887 8522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } } }
I20260809 23:18:47.233896 8482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:47.234006 8522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e [sys.catalog]: This master's current role is: FOLLOWER
W20260809 23:18:47.236012 8538 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:18:47.236104 8538 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:18:47.236203 8495 sys_catalog.cc:455] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } } }
I20260809 23:18:47.236302 8495 sys_catalog.cc:458] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:18:47.235966 8368 master_runner.cc:186] Error getting master registration for 127.2.76.254:43285: OK, instance_id { permanent_uuid: "b5013caacf814af79070dba4aec7a316" instance_seqno: 1786317526926880 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:18:47.237730 8368 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:18:47.265146 8223 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } attempt: 2
Aug 09 23:18:47 dist-test-slave-7vjv krb5kdc[8078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317526, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for kudu/127.2.76.252@KRBTEST.COM
I20260809 23:18:47.271737 8491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.265440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54688 (local address 127.2.76.252:34517)
0809 23:18:47.265470 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.265471 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.265479 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:18:47.265516 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.265516 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.265529 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.265584 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.265845 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.266471 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.267303 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.267558 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.267611 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.269364 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:47.269365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:47.269366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:47.269371 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:47.270663 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.271033 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.271034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.271034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.271087 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.271273 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.271274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.271274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.271380 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:47.271434 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:47.271600 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 23:18:47.271663 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 23:18:47.271737 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.265373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:34517 (local address 127.0.0.1:54688)
0809 23:18:47.265414 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:47.265424 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:47.265459 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:47.265593 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:47.265593 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:47.265755 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:47.265791 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.265792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.266537 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.266538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.267226 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:47.267228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.267621 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.267622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:47.267669 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.268063 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:47.268066 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:47.269286 (+ 1220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:47.270760 (+ 1474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.270761 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.270761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.270954 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.271154 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:47.271155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:47.271155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:47.271211 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:47.271432 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:47.271433 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:47.271472 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 23:18:47.271563 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 23:18:47.271653 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 23:18:47.273315 8223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316: Bootstrap starting.
I20260809 23:18:47.273939 8223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:47.274194 8223 log.cc:824] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:47.275089 8223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316: No bootstrap required, opened a new log
I20260809 23:18:47.276449 8223 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } }
I20260809 23:18:47.276589 8223 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5013caacf814af79070dba4aec7a316, State: Initialized, Role: FOLLOWER
I20260809 23:18:47.276697 8223 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } }
I20260809 23:18:47.276993 8223 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:47.277034 8178 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 23:18:47.277087 8178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:47.277832 8178 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } }
I20260809 23:18:47.278076 8178 leader_election.cc:288] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [CANDIDATE]: Term 1 election: Requested vote from peers c98607b3aa694cf7a560f6984fc8888b (127.2.76.253:38687), 6b5ae70642d9477bb7da77226258937e (127.2.76.252:34517)
I20260809 23:18:47.278585 8312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5013caacf814af79070dba4aec7a316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c98607b3aa694cf7a560f6984fc8888b"
I20260809 23:18:47.278687 8312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:47.279681 8312 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5013caacf814af79070dba4aec7a316 in term 1.
I20260809 23:18:47.281251 8123 leader_election.cc:302] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5013caacf814af79070dba4aec7a316, c98607b3aa694cf7a560f6984fc8888b; no voters:
I20260809 23:18:47.281790 8425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b5013caacf814af79070dba4aec7a316" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6b5ae70642d9477bb7da77226258937e"
I20260809 23:18:47.281908 8425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:47.282768 8425 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5013caacf814af79070dba4aec7a316 in term 1.
W20260809 23:18:47.279135 8561 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:18:47.283056 8561 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:18:47.276976 8542 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } } }
I20260809 23:18:47.283537 8099 master_runner.cc:263] No leader master found from master b5013caacf814af79070dba4aec7a316
I20260809 23:18:47.283578 8099 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:18:47.283658 8542 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:18:47.283743 8542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:47.283836 8542 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [term 1 LEADER]: Becoming Leader. State: Replica: b5013caacf814af79070dba4aec7a316, State: Running, Role: LEADER
I20260809 23:18:47.283941 8542 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } }
I20260809 23:18:47.284507 8542 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5013caacf814af79070dba4aec7a316. Latest consensus state: current_term: 1 leader_uuid: "b5013caacf814af79070dba4aec7a316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } } }
I20260809 23:18:47.284629 8542 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:47.285853 8569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:47.286073 8569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:47.305680 8425 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e [term 1 FOLLOWER]: Refusing update from remote peer b5013caacf814af79070dba4aec7a316: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:47.305893 8312 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b [term 1 FOLLOWER]: Refusing update from remote peer b5013caacf814af79070dba4aec7a316: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:47.306340 8567 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:47.306483 8567 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:47.307682 8495 sys_catalog.cc:455] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5013caacf814af79070dba4aec7a316. Latest consensus state: current_term: 1 leader_uuid: "b5013caacf814af79070dba4aec7a316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } } }
I20260809 23:18:47.307761 8495 sys_catalog.cc:458] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:18:47.308717 8522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5013caacf814af79070dba4aec7a316. Latest consensus state: current_term: 1 leader_uuid: "b5013caacf814af79070dba4aec7a316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } } }
I20260809 23:18:47.308804 8522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:18:47.310511 8491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.284989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54690 (local address 127.2.76.252:34517)
0809 23:18:47.285035 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.285037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.287250 (+ 2213us) server_negotiation.cc:407] Connection header received
0809 23:18:47.288197 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.288198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.288207 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.288231 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.289684 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.290309 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.294552 (+ 4243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.294805 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.294839 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.299692 (+ 4853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:47.299693 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:47.299694 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:47.299700 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:47.301209 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.308955 (+ 7746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.308956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.308956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.309021 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.309427 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.309428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.309428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.309533 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:47.309607 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:47.310368 (+ 761us) server_negotiation.cc:299] Negotiation successful
0809 23:18:47.310438 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 23:18:47.310952 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.285213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60316 (local address 127.2.76.254:43285)
0809 23:18:47.285255 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.285257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.287173 (+ 1916us) server_negotiation.cc:407] Connection header received
0809 23:18:47.287377 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.287378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.287387 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.287412 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.291222 (+ 3810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.291835 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.295255 (+ 3420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.295505 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.295541 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.301549 (+ 6008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:47.301550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:47.301550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:47.301556 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:47.303072 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.309224 (+ 6152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.309225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.309225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.309267 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.309893 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.309894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.309894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.310014 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:47.310093 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:47.310812 (+ 719us) server_negotiation.cc:299] Negotiation successful
0809 23:18:47.310879 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:18:47.312647 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:47.285615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40790 (local address 127.2.76.253:38687)
0809 23:18:47.285648 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.285650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.288652 (+ 3002us) server_negotiation.cc:407] Connection header received
0809 23:18:47.289077 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.289078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.289089 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.289116 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.293062 (+ 3946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.293679 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.296882 (+ 3203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:47.297169 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:47.297205 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:47.303452 (+ 6247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:47.303453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:47.303454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:47.303459 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:47.304922 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.309766 (+ 4844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.309767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.309767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.309821 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:47.311322 (+ 1501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:47.311323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:47.311324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:47.311445 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:47.311510 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:47.312488 (+ 978us) server_negotiation.cc:299] Negotiation successful
0809 23:18:47.312569 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 23:18:47.314137 8272 tablet.cc:2369] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:18:47.314316 8542 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5013caacf814af79070dba4aec7a316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } } }
I20260809 23:18:47.314379 8542 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:47.314786 8495 sys_catalog.cc:455] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5013caacf814af79070dba4aec7a316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } } }
I20260809 23:18:47.314851 8495 sys_catalog.cc:458] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:18:47.315104 8574 mvcc.cc:205] Tried to move back new op lower bound from 7316756591818227712 to 7316756591756365824. Current Snapshot: MvccSnapshot[applied={T|T < 7316756591818227712}]
I20260809 23:18:47.315214 8575 mvcc.cc:205] Tried to move back new op lower bound from 7316756591818227712 to 7316756591756365824. Current Snapshot: MvccSnapshot[applied={T|T < 7316756591818227712}]
I20260809 23:18:47.316610 8569 catalog_manager.cc:1347] Generated new cluster ID: a6b5f517e15a4ad6be4ddc33371dbae8
I20260809 23:18:47.316651 8569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:47.316944 8567 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5013caacf814af79070dba4aec7a316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } } }
I20260809 23:18:47.317028 8567 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316 [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:47.317580 8577 mvcc.cc:205] Tried to move back new op lower bound from 7316756591818227712 to 7316756591756365824. Current Snapshot: MvccSnapshot[applied={T|T < 7316756591818227712}]
I20260809 23:18:47.320178 8522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5013caacf814af79070dba4aec7a316" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5013caacf814af79070dba4aec7a316" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 43285 } } peers { permanent_uuid: "c98607b3aa694cf7a560f6984fc8888b" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 38687 } } peers { permanent_uuid: "6b5ae70642d9477bb7da77226258937e" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 34517 } } }
I20260809 23:18:47.320248 8522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:18:47.328624 8569 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:47.329968 8569 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:18:47.334148 8569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5013caacf814af79070dba4aec7a316: Generated new TSK 0
I20260809 23:18:47.334352 8569 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 23:18:47.363693 8226 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:47.362510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60322 (local address 127.2.76.254:43285)
0809 23:18:47.362547 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.362549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.363088 (+ 539us) server_negotiation.cc:407] Connection header received
0809 23:18:47.363186 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.363187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.363197 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.363224 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.363632 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60322: BlockingRecv error: recv got EOF from 127.0.0.1:60322 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 23:18:47.364562 8491 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:47.362850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54702 (local address 127.2.76.252:34517)
0809 23:18:47.362883 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.362885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.364056 (+ 1171us) server_negotiation.cc:407] Connection header received
0809 23:18:47.364166 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.364167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.364177 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.364204 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.364502 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54702: BlockingRecv error: recv got EOF from 127.0.0.1:54702 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 23:18:47.365622 8493 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:47.364975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40792 (local address 127.2.76.253:38687)
0809 23:18:47.365025 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.365027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.365143 (+ 116us) server_negotiation.cc:407] Connection header received
0809 23:18:47.365231 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.365232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.365242 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.365270 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.365562 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40792: BlockingRecv error: recv got EOF from 127.0.0.1:40792 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 23:18:47.380285 8493 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:47.379723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40804 (local address 127.2.76.253:38687)
0809 23:18:47.379759 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.379761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.379769 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:18:47.379800 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.379801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.379817 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.379874 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.380231 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40804: BlockingRecv error: recv got EOF from 127.0.0.1:40804 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 23:18:47.404592 8493 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:47.392435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40814 (local address 127.2.76.253:38687)
0809 23:18:47.392471 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.392473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.392525 (+ 52us) server_negotiation.cc:407] Connection header received
0809 23:18:47.392604 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.392605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.392615 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.392666 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.404537 (+ 11871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40814: BlockingRecv error: recv got EOF from 127.0.0.1:40814 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 23:18:47.411896 8493 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:47.407816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40816 (local address 127.2.76.253:38687)
0809 23:18:47.407850 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.407852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.411360 (+ 3508us) server_negotiation.cc:407] Connection header received
0809 23:18:47.411398 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.411399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.411410 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.411469 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.411846 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40816: BlockingRecv error: recv got EOF from 127.0.0.1:40816 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 23:18:47.425465 8493 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:47.424748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40818 (local address 127.2.76.253:38687)
0809 23:18:47.424984 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.424986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.424994 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:18:47.425027 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.425028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.425036 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.425094 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.425420 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40818: BlockingRecv error: recv got EOF from 127.0.0.1:40818 (error 108)
Metrics: {"server-negotiator.queue_time_us":218}
W20260809 23:18:47.435338 8493 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:47.434697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40826 (local address 127.2.76.253:38687)
0809 23:18:47.434733 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.434735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.434835 (+ 100us) server_negotiation.cc:407] Connection header received
0809 23:18:47.434907 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.434908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.434917 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.434944 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.435283 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40826: BlockingRecv error: recv got EOF from 127.0.0.1:40826 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 23:18:47.452893 8493 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:47.452295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40838 (local address 127.2.76.253:38687)
0809 23:18:47.452332 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.452334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.452420 (+ 86us) server_negotiation.cc:407] Connection header received
0809 23:18:47.452479 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.452479 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.452493 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.452536 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.452850 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40838: BlockingRecv error: recv got EOF from 127.0.0.1:40838 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 23:18:47.486383 8493 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:47.485761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40850 (local address 127.2.76.253:38687)
0809 23:18:47.485826 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.485828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.485913 (+ 85us) server_negotiation.cc:407] Connection header received
0809 23:18:47.485987 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.485988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.485997 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.486065 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.486334 (+ 269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40850: BlockingRecv error: recv got EOF from 127.0.0.1:40850 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 23:18:47.551921 8493 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:47.551332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40862 (local address 127.2.76.253:38687)
0809 23:18:47.551388 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.551389 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.551475 (+ 86us) server_negotiation.cc:407] Connection header received
0809 23:18:47.551547 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.551547 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.551557 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.551625 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.551881 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40862: BlockingRecv error: recv got EOF from 127.0.0.1:40862 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 23:18:47.681640 8493 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:47.680926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40878 (local address 127.2.76.253:38687)
0809 23:18:47.681004 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.681006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.681102 (+ 96us) server_negotiation.cc:407] Connection header received
0809 23:18:47.681190 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.681190 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.681201 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.681269 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.681582 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40878: BlockingRecv error: recv got EOF from 127.0.0.1:40878 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 23:18:47.939306 8493 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:47.938762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40894 (local address 127.2.76.253:38687)
0809 23:18:47.938823 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:47.938824 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:47.938849 (+ 25us) server_negotiation.cc:407] Connection header received
0809 23:18:47.938900 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:47.938900 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:47.938909 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:47.938962 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:47.939244 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40894: BlockingRecv error: recv got EOF from 127.0.0.1:40894 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 23:18:48.226816 8511 catalog_manager.cc:1259] Loaded cluster ID: a6b5f517e15a4ad6be4ddc33371dbae8
I20260809 23:18:48.226876 8511 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b: loading cluster ID for follower catalog manager: success
I20260809 23:18:48.227600 8511 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b: acquiring CA information for follower catalog manager: success
I20260809 23:18:48.227893 8511 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c98607b3aa694cf7a560f6984fc8888b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:18:48.236478 8538 catalog_manager.cc:1259] Loaded cluster ID: a6b5f517e15a4ad6be4ddc33371dbae8
I20260809 23:18:48.236543 8538 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e: loading cluster ID for follower catalog manager: success
I20260809 23:18:48.237140 8538 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e: acquiring CA information for follower catalog manager: success
I20260809 23:18:48.237320 8538 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6b5ae70642d9477bb7da77226258937e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:18:48.454826 8721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:48.452690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40908 (local address 127.2.76.253:38687)
0809 23:18:48.452780 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:48.452782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:48.452790 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:18:48.452873 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:48.452873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:48.452914 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:48.452971 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:48.453130 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:48.453556 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:48.454182 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:48.454340 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:48.454383 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:48.454518 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:48.454616 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:48.454667 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:48.454701 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 23:18:48.454716 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":51,"threads_started":1}
I20260809 23:18:48.458312 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:48.456483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54712 (local address 127.2.76.252:34517)
0809 23:18:48.456584 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:48.456586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:48.456593 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:18:48.456626 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:48.456627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:48.456659 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:48.456716 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:48.456791 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:48.457181 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:48.457739 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:48.457893 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:48.457934 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:48.458051 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:48.458124 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:48.458171 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:48.458202 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 23:18:48.458216 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":63,"threads_started":1}
I20260809 23:18:48.461764 8721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:48.459714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40920 (local address 127.2.76.253:38687)
0809 23:18:48.459759 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:48.459761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:48.459792 (+ 31us) server_negotiation.cc:407] Connection header received
0809 23:18:48.459856 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:48.459857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:48.459864 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:48.459906 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:48.460055 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:48.460470 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:48.461145 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:48.461354 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:48.461410 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:48.461554 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:48.461578 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:48.461629 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:48.461685 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 23:18:48.461704 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:18:48.462522 8737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:48.459628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60328 (local address 127.2.76.254:43285)
0809 23:18:48.459807 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:48.459809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:48.459899 (+ 90us) server_negotiation.cc:407] Connection header received
0809 23:18:48.459958 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:48.459959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:48.460005 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:48.460071 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:48.460238 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:48.460675 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:48.461556 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:48.461896 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:48.461946 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:48.462172 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:48.462274 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:48.462339 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:48.462398 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 23:18:48.462420 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":59,"threads_started":1}
I20260809 23:18:48.462831 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:48.459687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54720 (local address 127.2.76.252:34517)
0809 23:18:48.459760 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:48.459762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:48.459883 (+ 121us) server_negotiation.cc:407] Connection header received
0809 23:18:48.459973 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:48.459974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:48.459984 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:48.460054 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:48.460680 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:48.461265 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:48.462217 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:48.462396 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:48.462462 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:48.462604 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:48.462630 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:48.462673 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:48.462721 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 23:18:48.462739 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 23:18:48.463997 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 8099
I20260809 23:18:48.476948 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 8236
I20260809 23:18:48.485502 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 8368
2026-08-09T23:18:48Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2936 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:18:48 dist-test-slave-7vjv krb5kdc[8746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:18:48 dist-test-slave-7vjv krb5kdc[8746](info): set up 2 sockets
Aug 09 23:18:48 dist-test-slave-7vjv krb5kdc[8746](info): commencing operation
krb5kdc: starting...
W20260809 23:18:49.611509 2355 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/test-user.keytab.
Aug 09 23:18:49 dist-test-slave-7vjv krb5kdc[8746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:18:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:18:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.76.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
Entry for principal kudu/127.2.76.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab.
I20260809 23:18:49.716655 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:39259
--webserver_interface=127.2.76.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:39259,127.2.76.253:44163,127.2.76.252:39377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:18:49.788161 8771 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:18:49.788327 8771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:49.788343 8771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:49.788360 8771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:49.789348 8771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:49.789378 8771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:49.789391 8771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:49.789402 8771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:49.789413 8771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:49.790617 8771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:39259,127.2.76.253:44163,127.2.76.252:39377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.254.keytab
--principal=kudu/127.2.76.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.254:39259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.76.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:49.790772 8771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:49.790907 8771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:49.795262 8782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:49.795281 8785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:49.795455 8771 server_base.cc:1034] running on GCE node
W20260809 23:18:49.795267 8781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:49.795761 8771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:49.796041 8771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:49.797196 8771 hybrid_clock.cc:648] HybridClock initialized: now 1786317529797184 us; error 27 us; skew 500 ppm
Aug 09 23:18:49 dist-test-slave-7vjv krb5kdc[8746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317529, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:49.799629 8771 init.cc:377] Logged in from keytab as kudu/127.2.76.254@KRBTEST.COM (short username kudu)
I20260809 23:18:49.799968 8771 webserver.cc:460] Webserver started at http://127.2.76.254:32855/ using document root <none> and password file <none>
I20260809 23:18:49.800184 8771 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:49.800227 8771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:49.800303 8771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:49.801105 8771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/data/instance:
uuid: "92468e175c784e18bfb1187be08ba87c"
format_stamp: "Formatted at 2026-08-09 23:18:49 on dist-test-slave-7vjv"
server_key: "62698142d228ed8b0c9a8f360aa1c772"
server_key_iv: "68dccff294d8ce68a52d0333163df18e"
server_key_version: "encryptionkey@0"
I20260809 23:18:49.801363 8771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/wal/instance:
uuid: "92468e175c784e18bfb1187be08ba87c"
format_stamp: "Formatted at 2026-08-09 23:18:49 on dist-test-slave-7vjv"
server_key: "62698142d228ed8b0c9a8f360aa1c772"
server_key_iv: "68dccff294d8ce68a52d0333163df18e"
server_key_version: "encryptionkey@0"
I20260809 23:18:49.802506 8771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:18:49.803112 8797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:49.803244 8771 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:49.803318 8771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/wal
uuid: "92468e175c784e18bfb1187be08ba87c"
format_stamp: "Formatted at 2026-08-09 23:18:49 on dist-test-slave-7vjv"
server_key: "62698142d228ed8b0c9a8f360aa1c772"
server_key_iv: "68dccff294d8ce68a52d0333163df18e"
server_key_version: "encryptionkey@0"
I20260809 23:18:49.803365 8771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:49.811614 8771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:49.813308 8771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:49.813413 8771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:49.816843 8771 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.254:39259
I20260809 23:18:49.816900 8902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.254:39259 every 8 connection(s)
I20260809 23:18:49.817162 8771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/data/info.pb
I20260809 23:18:49.818060 8905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:49.819255 8905 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:18:49.820502 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 8771
I20260809 23:18:49.820614 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-0/wal/instance
I20260809 23:18:49.820818 2355 external_mini_cluster.cc:1428] Setting key 4843ab68f802c7a126b0a51c208bed58
Aug 09 23:18:49 dist-test-slave-7vjv krb5kdc[8746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317529, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.76.253@KRBTEST.COM; defaulting to no policy
I20260809 23:18:49.828902 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:49.819866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:39259 (local address 127.0.0.1:47744)
0809 23:18:49.820023 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:49.820056 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:49.820624 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:49.820920 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:49.820923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:49.821128 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:49.821252 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:49.821255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:49.822568 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:49.822569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:49.823675 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:49.823678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:49.824190 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:49.824192 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:49.824264 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:49.824652 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:49.824661 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:49.826078 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:49.827551 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:49.827557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:49.827559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:49.827728 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:49.827945 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:49.827945 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:49.827945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:49.828045 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:49.828373 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:49.828378 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:49.828435 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 23:18:49.828537 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 23:18:49.828739 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":90,"thread_start_us":90,"threads_started":1}
I20260809 23:18:49.828905 8911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:49.819900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47744 (local address 127.2.76.254:39259)
0809 23:18:49.820036 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:49.820044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:49.820148 (+ 104us) server_negotiation.cc:407] Connection header received
0809 23:18:49.820801 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:49.820803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:49.820820 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:49.820895 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:49.821349 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:49.822426 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:49.823771 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:49.824055 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:49.824197 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:49.826157 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:49.826164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:49.826164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:49.826170 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:49.827459 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:49.827802 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:49.827804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:49.827804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:49.827878 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:49.828123 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:49.828124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:49.828125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:49.828302 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:49.828389 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:49.828571 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 23:18:49.828693 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260809 23:18:49.829594 8905 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:18:49.829871 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:49.829771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:44163 (local address 127.0.0.1:44904)
0809 23:18:49.829816 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:49.829854 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:44163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 23:18:49.829999 8801 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.76.253:44163: connect: Connection refused (error 111)
I20260809 23:18:49.830328 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:49.830243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:44163 (local address 127.0.0.1:44910)
0809 23:18:49.830288 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:49.830310 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:44163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 23:18:49.830442 8905 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.253:44163: Network error: Client connection negotiation failed: client connection to 127.2.76.253:44163: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.253.keytab.
Entry for principal kudu/127.2.76.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.253.keytab.
I20260809 23:18:49.843838 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.253:44163
--webserver_interface=127.2.76.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.253.keytab
--principal=kudu/127.2.76.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:39259,127.2.76.253:44163,127.2.76.252:39377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:18:49.879652 8905 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } attempt: 1
I20260809 23:18:49.879981 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:49.879860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:44163 (local address 127.0.0.1:44914)
0809 23:18:49.879910 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:49.879943 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:44163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 23:18:49.880371 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:49.880280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:44163 (local address 127.0.0.1:44926)
0809 23:18:49.880331 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:49.880352 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:44163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 23:18:49.880448 8905 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.253:44163: Network error: Client connection negotiation failed: client connection to 127.2.76.253:44163: connect: Connection refused (error 111)
W20260809 23:18:49.916682 8918 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:18:49.916868 8918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:49.916904 8918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:49.916936 8918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:49.917945 8918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:49.918015 8918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:49.918042 8918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:49.918068 8918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:49.918090 8918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:49.919443 8918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:39259,127.2.76.253:44163,127.2.76.252:39377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.253.keytab
--principal=kudu/127.2.76.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.253:44163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.76.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:49.919709 8918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:49.919929 8918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:49.924335 8929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:49.924417 8934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:49.924352 8937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:49.924384 8918 server_base.cc:1034] running on GCE node
I20260809 23:18:49.924753 8918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:49.925006 8918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:49.926185 8918 hybrid_clock.cc:648] HybridClock initialized: now 1786317529926171 us; error 24 us; skew 500 ppm
Aug 09 23:18:49 dist-test-slave-7vjv krb5kdc[8746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317529, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:49.928362 8918 init.cc:377] Logged in from keytab as kudu/127.2.76.253@KRBTEST.COM (short username kudu)
I20260809 23:18:49.928732 8918 webserver.cc:460] Webserver started at http://127.2.76.253:43999/ using document root <none> and password file <none>
I20260809 23:18:49.928926 8918 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:49.928990 8918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:49.929129 8918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:49.930042 8918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/data/instance:
uuid: "500705a149e64dc6b463416e1c148ac3"
format_stamp: "Formatted at 2026-08-09 23:18:49 on dist-test-slave-7vjv"
server_key: "3c168a628a547d8edd44a90d5f31450d"
server_key_iv: "820412008435c147931e4cd0e813e349"
server_key_version: "encryptionkey@0"
I20260809 23:18:49.930378 8918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/wal/instance:
uuid: "500705a149e64dc6b463416e1c148ac3"
format_stamp: "Formatted at 2026-08-09 23:18:49 on dist-test-slave-7vjv"
server_key: "3c168a628a547d8edd44a90d5f31450d"
server_key_iv: "820412008435c147931e4cd0e813e349"
server_key_version: "encryptionkey@0"
I20260809 23:18:49.931557 8918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 23:18:49.932248 8948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:49.932415 8918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:49.932507 8918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/wal
uuid: "500705a149e64dc6b463416e1c148ac3"
format_stamp: "Formatted at 2026-08-09 23:18:49 on dist-test-slave-7vjv"
server_key: "3c168a628a547d8edd44a90d5f31450d"
server_key_iv: "820412008435c147931e4cd0e813e349"
server_key_version: "encryptionkey@0"
I20260809 23:18:49.932587 8918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:49.947369 8918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:49.948588 8905 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } attempt: 2
I20260809 23:18:49.948930 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:49.948809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:44163 (local address 127.0.0.1:44936)
0809 23:18:49.948874 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:49.948905 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:44163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 23:18:49.949242 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:49.949147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:44163 (local address 127.0.0.1:44950)
0809 23:18:49.949194 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:49.949223 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.253:44163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 23:18:49.949292 8918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
W20260809 23:18:49.949326 8905 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.253:44163: Network error: Client connection negotiation failed: client connection to 127.2.76.253:44163: connect: Connection refused (error 111)
I20260809 23:18:49.949448 8918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:49.952878 8918 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.253:44163
I20260809 23:18:49.952937 9045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.253:44163 every 8 connection(s)
I20260809 23:18:49.953231 8918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/data/info.pb
I20260809 23:18:49.954118 9048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:49.955861 9048 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:18:49.957659 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 8918
I20260809 23:18:49.957791 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-1/wal/instance
I20260809 23:18:49.958004 2355 external_mini_cluster.cc:1428] Setting key 163ca048a07e57a4f76e8327751b6f27
Aug 09 23:18:49 dist-test-slave-7vjv krb5kdc[8746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317529, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.253@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:49.964591 8911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:49.956450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47752 (local address 127.2.76.254:39259)
0809 23:18:49.956483 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:49.956485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:49.956681 (+ 196us) server_negotiation.cc:407] Connection header received
0809 23:18:49.957061 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:49.957062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:49.957071 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:49.957120 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:49.957712 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:49.958254 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:49.959575 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:49.959772 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:49.959840 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:49.961864 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:49.961866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:49.961866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:49.961874 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:49.963298 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:49.963667 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:49.963668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:49.963668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:49.963725 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:49.963942 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:49.963943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:49.963943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:49.964086 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:49.964167 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:49.964435 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 23:18:49.964510 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:18:49.964763 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:49.956418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:39259 (local address 127.0.0.1:47752)
0809 23:18:49.956568 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:49.956604 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:49.956982 (+ 378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:49.957187 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:49.957190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:49.957385 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:49.957622 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:49.957629 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:49.958335 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:49.958337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:49.959479 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:49.959483 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:49.959880 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:49.959881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:49.959996 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:49.960498 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:49.960507 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:49.961756 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:49.963390 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:49.963397 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:49.963399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:49.963596 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:49.963787 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:49.963788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:49.963788 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:49.963881 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:49.964174 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:49.964181 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:49.964312 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 23:18:49.964402 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 23:18:49.964556 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":91,"threads_started":1}
I20260809 23:18:49.965306 9048 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.76.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.76.252@KRBTEST.COM" created.
Aug 09 23:18:49 dist-test-slave-7vjv krb5kdc[8746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317529, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.253@KRBTEST.COM for kudu/127.2.76.253@KRBTEST.COM
I20260809 23:18:49.972406 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:49.965570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:44163 (local address 127.0.0.1:44960)
0809 23:18:49.965638 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:49.965649 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:49.965691 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:49.965910 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:49.965911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:49.966052 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:49.966106 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:49.966108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:49.966887 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:49.966888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:49.967363 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:49.967364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:49.967762 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:49.967763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:49.967811 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:49.968239 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:49.968243 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:49.969482 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:49.971421 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:49.971422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:49.971422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:49.971573 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:49.971793 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:49.971793 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:49.971794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:49.971843 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:49.972125 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:49.972126 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:49.972162 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 23:18:49.972222 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 23:18:49.972313 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 23:18:49.972569 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:49.965628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44960 (local address 127.2.76.253:44163)
0809 23:18:49.965769 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:49.965773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:49.965789 (+ 16us) server_negotiation.cc:407] Connection header received
0809 23:18:49.965808 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:49.965812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:49.965825 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:49.965882 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:49.966177 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:49.966816 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:49.967436 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:49.967699 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:49.967750 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:49.969579 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:49.969586 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:49.969586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:49.969604 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:49.971334 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:49.971660 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:49.971663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:49.971663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:49.971713 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:49.971910 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:49.971911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:49.971911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:49.972037 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:49.972131 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:49.972267 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 23:18:49.972375 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":115,"thread_start_us":93,"threads_started":1}
I20260809 23:18:49.973027 9048 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:18:49.973328 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:49.973215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:39377 (local address 127.0.0.1:44874)
0809 23:18:49.973264 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:49.973302 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 23:18:49.973474 8955 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
I20260809 23:18:49.973858 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:49.973757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:39377 (local address 127.0.0.1:44888)
0809 23:18:49.973806 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:49.973828 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 23:18:49.973978 9048 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.252:39377: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.76.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.252.keytab.
Entry for principal kudu/127.2.76.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.252.keytab.
I20260809 23:18:49.983026 2355 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvJxnr9/build/release/bin/kudu
/tmp/dist-test-taskvJxnr9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.252:39377
--webserver_interface=127.2.76.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.252.keytab
--principal=kudu/127.2.76.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.76.212:45885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.76.254:39259,127.2.76.253:44163,127.2.76.252:39377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:18:50.023165 9048 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } attempt: 1
I20260809 23:18:50.023540 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.023424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:39377 (local address 127.0.0.1:44904)
0809 23:18:50.023475 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:50.023514 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 23:18:50.023840 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.023757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:39377 (local address 127.0.0.1:44916)
0809 23:18:50.023797 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:50.023817 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 23:18:50.023917 9048 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.252:39377: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
I20260809 23:18:50.040439 8905 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } attempt: 3
Aug 09 23:18:50 dist-test-slave-7vjv krb5kdc[8746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317529, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for kudu/127.2.76.253@KRBTEST.COM
I20260809 23:18:50.047024 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.040734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:44163 (local address 127.0.0.1:44974)
0809 23:18:50.040790 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:50.040801 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:50.040854 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:50.040999 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:50.041004 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:50.041132 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:50.041195 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:50.041196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.042031 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.042032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:50.042470 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:50.042471 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.042928 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.042929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:50.042978 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.043388 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:50.043392 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:50.044476 (+ 1084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:50.046133 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:50.046134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:50.046135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:50.046275 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:50.046458 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:50.046458 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:50.046459 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:50.046503 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:50.046762 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:50.046762 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:50.046814 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 23:18:50.046884 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 23:18:50.046951 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 23:18:50.047109 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.040779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44974 (local address 127.2.76.253:44163)
0809 23:18:50.040850 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.040852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.040888 (+ 36us) server_negotiation.cc:407] Connection header received
0809 23:18:50.040917 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.040918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.040931 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.040980 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.041262 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.041944 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.042547 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.042865 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.042933 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.044565 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:50.044567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:50.044567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:50.044574 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:50.046052 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.046338 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.046339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.046339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.046397 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.046555 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.046556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.046557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.046700 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:50.046773 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:50.046924 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 23:18:50.047017 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 23:18:50.047739 8905 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:18:50.048074 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.047969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:39377 (local address 127.0.0.1:44922)
0809 23:18:50.048021 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:50.048053 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 23:18:50.048405 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.048342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:39377 (local address 127.0.0.1:44936)
0809 23:18:50.048375 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:50.048394 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 23:18:50.048499 8905 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.252:39377: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
W20260809 23:18:50.059237 9059 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:18:50.059427 9059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:18:50.059458 9059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:18:50.059500 9059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:18:50.060552 9059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:18:50.060606 9059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:18:50.060628 9059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:18:50.060653 9059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:18:50.060678 9059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:18:50.061995 9059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.76.212:45885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.76.254:39259,127.2.76.253:44163,127.2.76.252:39377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/krb5kdc/kudu_127.2.76.252.keytab
--principal=kudu/127.2.76.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.76.252:39377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.76.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:18:50.062260 9059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:18:50.062455 9059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:18:50.066597 9072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:18:50.066682 9069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:50.066809 9059 server_base.cc:1034] running on GCE node
W20260809 23:18:50.066627 9074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:18:50.067126 9059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:18:50.067335 9059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:18:50.068461 9059 hybrid_clock.cc:648] HybridClock initialized: now 1786317530068451 us; error 22 us; skew 500 ppm
Aug 09 23:18:50 dist-test-slave-7vjv krb5kdc[8746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317530, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:18:50.070536 9059 init.cc:377] Logged in from keytab as kudu/127.2.76.252@KRBTEST.COM (short username kudu)
I20260809 23:18:50.070958 9059 webserver.cc:460] Webserver started at http://127.2.76.252:46233/ using document root <none> and password file <none>
I20260809 23:18:50.071174 9059 fs_manager.cc:359] Metadata directory not provided
I20260809 23:18:50.071235 9059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:18:50.071372 9059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:18:50.072316 9059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/data/instance:
uuid: "969f3bdda38a41ceb298a21a54c5e4da"
format_stamp: "Formatted at 2026-08-09 23:18:50 on dist-test-slave-7vjv"
server_key: "d7d1146e73e379d7bf7b02ec5db188c5"
server_key_iv: "c7c755ada218691d1f91802e4e8e00cd"
server_key_version: "encryptionkey@0"
I20260809 23:18:50.072640 9059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/wal/instance:
uuid: "969f3bdda38a41ceb298a21a54c5e4da"
format_stamp: "Formatted at 2026-08-09 23:18:50 on dist-test-slave-7vjv"
server_key: "d7d1146e73e379d7bf7b02ec5db188c5"
server_key_iv: "c7c755ada218691d1f91802e4e8e00cd"
server_key_version: "encryptionkey@0"
I20260809 23:18:50.073818 9059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:18:50.074425 9092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:18:50.074589 9059 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:18:50.074680 9059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/data,/tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/wal
uuid: "969f3bdda38a41ceb298a21a54c5e4da"
format_stamp: "Formatted at 2026-08-09 23:18:50 on dist-test-slave-7vjv"
server_key: "d7d1146e73e379d7bf7b02ec5db188c5"
server_key_iv: "c7c755ada218691d1f91802e4e8e00cd"
server_key_version: "encryptionkey@0"
I20260809 23:18:50.074750 9059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:18:50.079617 8905 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } attempt: 1
I20260809 23:18:50.079927 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.079829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:39377 (local address 127.0.0.1:44944)
0809 23:18:50.079876 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:50.079900 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 23:18:50.080279 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.080185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:39377 (local address 127.0.0.1:44952)
0809 23:18:50.080235 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:50.080257 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 23:18:50.080440 8905 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.252:39377: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
I20260809 23:18:50.088608 9059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:18:50.090277 9059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:18:50.090430 9059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:18:50.092038 9048 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } attempt: 2
I20260809 23:18:50.092429 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.092299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:39377 (local address 127.0.0.1:44954)
0809 23:18:50.092349 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:50.092383 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 23:18:50.092785 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.092686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:39377 (local address 127.0.0.1:44958)
0809 23:18:50.092716 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:50.092740 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260809 23:18:50.092883 9048 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.76.252:39377: Network error: Client connection negotiation failed: client connection to 127.2.76.252:39377: connect: Connection refused (error 111)
I20260809 23:18:50.094568 9059 rpc_server.cc:307] RPC server started. Bound to: 127.2.76.252:39377
I20260809 23:18:50.094595 9185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.76.252:39377 every 8 connection(s)
I20260809 23:18:50.094905 9059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/data/info.pb
I20260809 23:18:50.095765 9187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:18:50.097071 9187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:18:50.097379 2355 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvJxnr9/build/release/bin/kudu as pid 9059
I20260809 23:18:50.097497 2355 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvJxnr9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786317238298104-2355-0/minicluster-data/master-2/wal/instance
I20260809 23:18:50.097709 2355 external_mini_cluster.cc:1428] Setting key fdfb3e4459c953fd955128c6779ba2ef
Aug 09 23:18:50 dist-test-slave-7vjv krb5kdc[8746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317530, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.252@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
Aug 09 23:18:50 dist-test-slave-7vjv krb5kdc[8746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.252@KRBTEST.COM
I20260809 23:18:50.105867 8911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.097656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47762 (local address 127.2.76.254:39259)
0809 23:18:50.097697 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.097699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.097901 (+ 202us) server_negotiation.cc:407] Connection header received
0809 23:18:50.098478 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.098479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.098490 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.098554 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.098997 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.099435 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.100553 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.100839 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.100911 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.102729 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:50.102730 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:50.102730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:50.102736 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:50.104534 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.104998 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.104999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.105000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.105057 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.105293 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.105294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.105294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.105400 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:50.105468 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:50.105719 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 23:18:50.105795 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:18:50.106029 9190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.097653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.254:39259 (local address 127.0.0.1:47762)
0809 23:18:50.097787 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:50.097822 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:50.098373 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:50.098584 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:50.098586 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:50.098795 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:50.098918 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:50.098921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.099507 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.099508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:50.100478 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:50.100481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.100953 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.100954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:50.101070 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.101477 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:50.101484 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:50.102645 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:50.104614 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:50.104620 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:50.104621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:50.104902 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:50.105105 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:50.105106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:50.105107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:50.105221 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:50.105462 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:50.105469 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:50.105593 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 23:18:50.105687 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 23:18:50.105816 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":60,"thread_start_us":72,"threads_started":1}
I20260809 23:18:50.106205 9192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.098060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44964 (local address 127.2.76.252:39377)
0809 23:18:50.098218 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.098223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.098259 (+ 36us) server_negotiation.cc:407] Connection header received
0809 23:18:50.098373 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.098380 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.098401 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.098478 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.099076 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.100081 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.100924 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.101288 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.101333 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.103390 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:50.103396 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:50.103396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:50.103402 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:50.105006 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.105326 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.105329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.105329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.105385 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.105654 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.105655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.105655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.105816 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:50.105960 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:50.106027 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 23:18:50.106107 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":97,"threads_started":1}
I20260809 23:18:50.106691 9187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:18:50 dist-test-slave-7vjv krb5kdc[8746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317530, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.252@KRBTEST.COM for kudu/127.2.76.253@KRBTEST.COM
Aug 09 23:18:50 dist-test-slave-7vjv krb5kdc[8746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.254@KRBTEST.COM
I20260809 23:18:50.113446 9190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.106893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.253:44163 (local address 127.0.0.1:44986)
0809 23:18:50.106965 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:50.106977 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:50.107012 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:50.107124 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:50.107125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:50.107247 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:50.107277 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:50.107278 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.107932 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.107933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:50.108526 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:50.108528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.109348 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.109349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:50.109386 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.109616 (+ 230us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:50.109620 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:50.110736 (+ 1116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:50.112581 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:50.112583 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:50.112583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:50.112782 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:50.112979 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:50.112980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:50.112981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:50.113040 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:50.113238 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:50.113239 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:50.113279 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 23:18:50.113338 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 23:18:50.113400 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 23:18:50.113550 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.106943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44986 (local address 127.2.76.253:44163)
0809 23:18:50.106979 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.106981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.107003 (+ 22us) server_negotiation.cc:407] Connection header received
0809 23:18:50.107065 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.107066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.107075 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.107112 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.107339 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.107864 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.108601 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.109274 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.109336 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.110813 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:50.110814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:50.110815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:50.110821 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:50.112495 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.112858 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.112859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.112859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.112907 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.113100 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.113100 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.113101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.113183 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:50.113223 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:50.113381 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 23:18:50.113445 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:18:50.113889 9187 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:18:50.114341 8911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.106935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47774 (local address 127.2.76.254:39259)
0809 23:18:50.106967 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.106969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.106976 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:18:50.106995 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.106996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.107004 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.107055 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.107535 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.108053 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.108792 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.109069 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.109146 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.111726 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:50.111727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:50.111727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:50.111734 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:50.113223 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.113646 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.113647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.113648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.113697 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.113900 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.113901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.113902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.114019 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:50.114159 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:50.114197 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 23:18:50.114263 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
Aug 09 23:18:50 dist-test-slave-7vjv krb5kdc[8746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317530, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.252@KRBTEST.COM for kudu/127.2.76.252@KRBTEST.COM
Aug 09 23:18:50 dist-test-slave-7vjv krb5kdc[8746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.76.253@KRBTEST.COM
I20260809 23:18:50.119899 9190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.114103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:39377 (local address 127.0.0.1:44972)
0809 23:18:50.114128 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:50.114136 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:50.114169 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:50.114271 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:50.114271 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:50.114407 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:50.114434 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:50.114435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.115331 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.115332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:50.115770 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:50.115771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.116174 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.116175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:50.116222 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.116577 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:50.116580 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:50.117621 (+ 1041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:50.118907 (+ 1286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:50.118908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:50.118908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:50.119123 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:50.119308 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:50.119308 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:50.119309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:50.119360 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:50.119597 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:50.119598 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:50.119631 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 23:18:50.119696 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 23:18:50.119787 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13}
I20260809 23:18:50.119947 9192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.114131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44972 (local address 127.2.76.252:39377)
0809 23:18:50.114158 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.114159 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.114166 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:18:50.114216 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.114217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.114225 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.114268 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.114598 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.115232 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.115854 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.116067 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.116173 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.117695 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:50.117696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:50.117696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:50.117702 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:50.118834 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.119201 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.119202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.119202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.119246 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.119433 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.119433 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.119434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.119524 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:50.119604 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:50.119715 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 23:18:50.119845 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":16}
I20260809 23:18:50.121177 9187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da: Bootstrap starting.
I20260809 23:18:50.121755 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.114817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45000 (local address 127.2.76.253:44163)
0809 23:18:50.114847 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.114848 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.114889 (+ 41us) server_negotiation.cc:407] Connection header received
0809 23:18:50.114945 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.114946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.114952 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.115005 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.115563 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.115970 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.116784 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.117043 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.117109 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.119256 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:50.119258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:50.119258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:50.119264 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:50.120819 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.121131 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.121132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.121132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.121184 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.121341 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.121342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.121342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.121442 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:50.121587 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:50.121619 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 23:18:50.121677 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 23:18:50.121927 9187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:50.122234 9187 log.cc:824] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:50.122973 9187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da: No bootstrap required, opened a new log
I20260809 23:18:50.122970 2355 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:18:50.123677 8845 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "92468e175c784e18bfb1187be08ba87c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47774
I20260809 23:18:50.124302 9187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } }
I20260809 23:18:50.124445 9187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 969f3bdda38a41ceb298a21a54c5e4da, State: Initialized, Role: FOLLOWER
I20260809 23:18:50.124519 9187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } }
I20260809 23:18:50.124879 9196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } } }
I20260809 23:18:50.124922 9187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:50.124985 9196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da [sys.catalog]: This master's current role is: FOLLOWER
W20260809 23:18:50.126581 9215 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:18:50.126660 9215 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:18:50.126909 9059 master_runner.cc:186] Error getting master registration for 127.2.76.254:39259: OK, instance_id { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" instance_seqno: 1786317529815847 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:18:50.129132 9059 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:18:50.155537 8905 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } attempt: 2
Aug 09 23:18:50 dist-test-slave-7vjv krb5kdc[8746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317529, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.254@KRBTEST.COM for kudu/127.2.76.252@KRBTEST.COM
I20260809 23:18:50.161458 9192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.155851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44988 (local address 127.2.76.252:39377)
0809 23:18:50.155894 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.155896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.155905 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:18:50.155934 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.155934 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.155943 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.155982 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.156305 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.156683 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.157260 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.157423 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.157493 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.159142 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:50.159143 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:50.159143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:50.159149 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:50.160410 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.160762 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.160763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.160763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.160812 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.161006 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.161007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.161007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.161118 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:50.161183 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:50.161328 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 23:18:50.161389 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:18:50.161458 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.155784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:39377 (local address 127.0.0.1:44988)
0809 23:18:50.155842 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:50.155851 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:50.155888 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:50.155998 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:50.155999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:50.156193 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:50.156225 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:50.156227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.156753 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.156754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:50.157181 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:50.157183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.157501 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.157502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:50.157549 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.157933 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:50.157938 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:50.159060 (+ 1122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:50.160526 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:50.160527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:50.160528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:50.160684 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:50.160879 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:50.160879 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:50.160880 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:50.160930 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:50.161188 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:50.161189 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:50.161227 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 23:18:50.161296 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 23:18:50.161379 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 23:18:50.162864 8905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c: Bootstrap starting.
I20260809 23:18:50.163514 8905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:50.163785 8905 log.cc:824] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:50.164516 8905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c: No bootstrap required, opened a new log
I20260809 23:18:50.165741 8905 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } }
I20260809 23:18:50.165858 8905 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92468e175c784e18bfb1187be08ba87c, State: Initialized, Role: FOLLOWER
I20260809 23:18:50.165930 8905 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } }
I20260809 23:18:50.166188 8905 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:50.166177 9222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } } }
I20260809 23:18:50.166262 9222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:18:50.166298 8845 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 23:18:50.166381 8845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:50.167048 8845 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } }
I20260809 23:18:50.167699 8845 leader_election.cc:288] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [CANDIDATE]: Term 1 election: Requested vote from peers 500705a149e64dc6b463416e1c148ac3 (127.2.76.253:44163), 969f3bdda38a41ceb298a21a54c5e4da (127.2.76.252:39377)
I20260809 23:18:50.167807 9139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92468e175c784e18bfb1187be08ba87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "969f3bdda38a41ceb298a21a54c5e4da"
I20260809 23:18:50.167901 9139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da [term 0 FOLLOWER]: Advancing to term 1
W20260809 23:18:50.168457 9237 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:18:50.168556 9237 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:18:50.168933 9139 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92468e175c784e18bfb1187be08ba87c in term 1.
I20260809 23:18:50.169603 8771 master_runner.cc:186] Error getting master registration for 127.2.76.253:44163: OK, instance_id { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" instance_seqno: 1786317529951893 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:18:50.171639 8771 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:18:50.171794 8800 leader_election.cc:302] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92468e175c784e18bfb1187be08ba87c, 969f3bdda38a41ceb298a21a54c5e4da; no voters:
I20260809 23:18:50.171891 9222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:18:50.172008 9222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [term 1 LEADER]: Becoming Leader. State: Replica: 92468e175c784e18bfb1187be08ba87c, State: Running, Role: LEADER
I20260809 23:18:50.172082 9222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } }
I20260809 23:18:50.172430 9222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92468e175c784e18bfb1187be08ba87c. Latest consensus state: current_term: 1 leader_uuid: "92468e175c784e18bfb1187be08ba87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } } }
I20260809 23:18:50.172519 9222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:50.172691 9255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:18:50.172873 9255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:18:50.174604 9139 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da [term 1 FOLLOWER]: Refusing update from remote peer 92468e175c784e18bfb1187be08ba87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:50.174925 9222 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:50.176039 9196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92468e175c784e18bfb1187be08ba87c. Latest consensus state: current_term: 1 leader_uuid: "92468e175c784e18bfb1187be08ba87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } } }
I20260809 23:18:50.176151 9196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:18:50.177554 9222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92468e175c784e18bfb1187be08ba87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } } }
I20260809 23:18:50.177628 9222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:50.177943 9196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92468e175c784e18bfb1187be08ba87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } } }
I20260809 23:18:50.178002 9196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:18:50.178248 9256 mvcc.cc:205] Tried to move back new op lower bound from 7316756603592716288 to 7316756603585470464. Current Snapshot: MvccSnapshot[applied={T|T < 7316756603592716288}]
I20260809 23:18:50.178783 9257 mvcc.cc:205] Tried to move back new op lower bound from 7316756603592716288 to 7316756603585470464. Current Snapshot: MvccSnapshot[applied={T|T < 7316756603592716288}]
I20260809 23:18:50.184021 9048 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } attempt: 3
I20260809 23:18:50.184484 9255 catalog_manager.cc:1347] Generated new cluster ID: 9e4a9ec0f1404709883f3f22df5307da
I20260809 23:18:50.184531 9255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:18:50.194581 9192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.170827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44998 (local address 127.2.76.252:39377)
0809 23:18:50.170857 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.170859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.172339 (+ 1480us) server_negotiation.cc:407] Connection header received
0809 23:18:50.173059 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.173060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.173080 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.173107 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.176429 (+ 3322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.177062 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.182192 (+ 5130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.182449 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.182483 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.187226 (+ 4743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:50.187227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:50.187227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:50.187233 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:50.188824 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.192598 (+ 3774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.192599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.192599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.192658 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.193476 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.193477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.193477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.193607 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:50.193685 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:50.194436 (+ 751us) server_negotiation.cc:299] Negotiation successful
0809 23:18:50.194504 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 23:18:50.197453 8911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.169967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47784 (local address 127.2.76.254:39259)
0809 23:18:50.170010 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.170012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.171241 (+ 1229us) server_negotiation.cc:407] Connection header received
0809 23:18:50.171365 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.171366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.171375 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.171400 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.174822 (+ 3422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.175446 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.181541 (+ 6095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.181826 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.181883 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.189772 (+ 7889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:50.189773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:50.189773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:50.189779 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:50.191207 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.192255 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.192256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.192257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.192339 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.193748 (+ 1409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.193749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.193750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.193868 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:50.193931 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:50.197294 (+ 3363us) server_negotiation.cc:299] Negotiation successful
0809 23:18:50.197362 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 09 23:18:50 dist-test-slave-7vjv krb5kdc[8746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317529, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.76.253@KRBTEST.COM for kudu/127.2.76.252@KRBTEST.COM
I20260809 23:18:50.199668 9056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.170394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45016 (local address 127.2.76.253:44163)
0809 23:18:50.170430 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.170432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.172157 (+ 1725us) server_negotiation.cc:407] Connection header received
0809 23:18:50.172626 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.172627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.172646 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.172672 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.177514 (+ 4842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.178108 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.182677 (+ 4569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.182943 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.182978 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.187481 (+ 4503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:50.187482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:50.187483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:50.187489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:50.189505 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.192446 (+ 2941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.192447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.192447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.192506 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.193207 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.193208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.193208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.193339 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:50.193414 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:50.199509 (+ 6095us) server_negotiation.cc:299] Negotiation successful
0809 23:18:50.199591 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:18:50.202232 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.184248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.76.252:39377 (local address 127.0.0.1:45002)
0809 23:18:50.184316 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 23:18:50.184324 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:18:50.184357 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:18:50.194861 (+ 10504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:18:50.194862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:18:50.195004 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:18:50.195038 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:50.195040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.195882 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.195883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:50.196583 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:18:50.196586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.197041 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.197042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:18:50.197103 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.197766 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:18:50.197781 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:18:50.199078 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:18:50.201170 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:50.201171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:50.201171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:50.201359 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:50.201607 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:18:50.201607 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:18:50.201608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:18:50.201657 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:18:50.201975 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:18:50.201976 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:18:50.202030 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 23:18:50.202079 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 23:18:50.202155 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 23:18:50.202782 9192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:50.184423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45002 (local address 127.2.76.252:39377)
0809 23:18:50.194638 (+ 10215us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.194640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.194646 (+ 6us) server_negotiation.cc:407] Connection header received
0809 23:18:50.194661 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.194662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.194669 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.194715 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.195204 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.195789 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.196680 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:50.196932 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:50.196980 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:50.199171 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:18:50.199172 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:18:50.199172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:18:50.199178 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:18:50.201076 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.201437 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.201437 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.201438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.201507 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:18:50.201720 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:18:50.201721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:18:50.201721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:18:50.201872 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:18:50.201943 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:50.202579 (+ 636us) server_negotiation.cc:299] Negotiation successful
0809 23:18:50.202666 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10204,"thread_start_us":4000,"threads_started":1}
I20260809 23:18:50.204269 9048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3: Bootstrap starting.
I20260809 23:18:50.204879 9048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3: Neither blocks nor log segments found. Creating new log.
I20260809 23:18:50.205148 9048 log.cc:824] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3: Log is configured to *not* fsync() on all Append() calls
I20260809 23:18:50.207381 9048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3: No bootstrap required, opened a new log
I20260809 23:18:50.209144 9048 raft_consensus.cc:348] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } }
I20260809 23:18:50.209255 9048 raft_consensus.cc:729] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500705a149e64dc6b463416e1c148ac3, State: Initialized, Role: FOLLOWER
I20260809 23:18:50.209312 9048 consensus_queue.cc:260] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } }
I20260809 23:18:50.209570 9048 sys_catalog.cc:564] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:18:50.209650 8998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:18:50.210341 8998 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3 [term 1 FOLLOWER]: Refusing update from remote peer 92468e175c784e18bfb1187be08ba87c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:18:50.210543 9002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "92468e175c784e18bfb1187be08ba87c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "500705a149e64dc6b463416e1c148ac3"
I20260809 23:18:50.211274 9002 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92468e175c784e18bfb1187be08ba87c in term 1.
I20260809 23:18:50.211541 9261 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [LEADER]: Connected to new peer: Peer: permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:18:50.213292 9255 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:18:50.213872 9255 catalog_manager.cc:1504] Loading token signing keys...
W20260809 23:18:50.214812 9286 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:18:50.214905 9286 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:18:50.215538 9249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92468e175c784e18bfb1187be08ba87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } } }
I20260809 23:18:50.215623 9249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c [sys.catalog]: This master's current role is: LEADER
I20260809 23:18:50.216253 9288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92468e175c784e18bfb1187be08ba87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } } }
I20260809 23:18:50.216372 9288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:18:50.216466 9288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92468e175c784e18bfb1187be08ba87c. Latest consensus state: current_term: 1 leader_uuid: "92468e175c784e18bfb1187be08ba87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } } }
I20260809 23:18:50.216560 9288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:18:50.216620 9288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92468e175c784e18bfb1187be08ba87c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92468e175c784e18bfb1187be08ba87c" member_type: VOTER last_known_addr { host: "127.2.76.254" port: 39259 } } peers { permanent_uuid: "500705a149e64dc6b463416e1c148ac3" member_type: VOTER last_known_addr { host: "127.2.76.253" port: 44163 } } peers { permanent_uuid: "969f3bdda38a41ceb298a21a54c5e4da" member_type: VOTER last_known_addr { host: "127.2.76.252" port: 39377 } } }
I20260809 23:18:50.216692 9288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:18:50.217211 9290 mvcc.cc:205] Tried to move back new op lower bound from 7316756603592716288 to 7316756603585470464. Current Snapshot: MvccSnapshot[applied={T|T < 7316756603592716288}]
I20260809 23:18:50.229734 9255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92468e175c784e18bfb1187be08ba87c: Generated new TSK 0
I20260809 23:18:50.229882 9255 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 23:18:50.247543 8911 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:50.246065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47798 (local address 127.2.76.254:39259)
0809 23:18:50.246101 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.246103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.246904 (+ 801us) server_negotiation.cc:407] Connection header received
0809 23:18:50.247016 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.247018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.247028 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.247058 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.247472 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47798: BlockingRecv error: recv got EOF from 127.0.0.1:47798 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 23:18:50.248528 9266 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:50.246615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45012 (local address 127.2.76.252:39377)
0809 23:18:50.246652 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.246653 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.247862 (+ 1209us) server_negotiation.cc:407] Connection header received
0809 23:18:50.248000 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.248001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.248012 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.248055 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.248452 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45012: BlockingRecv error: recv got EOF from 127.0.0.1:45012 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 23:18:50.248939 9056 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:50.246347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45030 (local address 127.2.76.253:44163)
0809 23:18:50.246377 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.246379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.247740 (+ 1361us) server_negotiation.cc:407] Connection header received
0809 23:18:50.248152 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.248153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.248162 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.248185 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.248894 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45030: BlockingRecv error: recv got EOF from 127.0.0.1:45030 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 23:18:50.250933 9056 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:50.250249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45032 (local address 127.2.76.253:44163)
0809 23:18:50.250277 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.250279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.250471 (+ 192us) server_negotiation.cc:407] Connection header received
0809 23:18:50.250579 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.250580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.250594 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.250619 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.250896 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45032: BlockingRecv error: recv got EOF from 127.0.0.1:45032 (error 108)
Metrics: {"server-negotiator.queue_time_us":14}
W20260809 23:18:50.253605 9056 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:50.252902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45046 (local address 127.2.76.253:44163)
0809 23:18:50.252930 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.252932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.253145 (+ 213us) server_negotiation.cc:407] Connection header received
0809 23:18:50.253229 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.253230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.253238 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.253280 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.253569 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45046: BlockingRecv error: recv got EOF from 127.0.0.1:45046 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 23:18:50.257164 9056 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:50.256547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45052 (local address 127.2.76.253:44163)
0809 23:18:50.256574 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.256575 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.256774 (+ 199us) server_negotiation.cc:407] Connection header received
0809 23:18:50.256850 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.256851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.256858 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.256881 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.257128 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45052: BlockingRecv error: recv got EOF from 127.0.0.1:45052 (error 108)
Metrics: {"server-negotiator.queue_time_us":13}
W20260809 23:18:50.262766 9056 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:50.262071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45054 (local address 127.2.76.253:44163)
0809 23:18:50.262102 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.262103 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.262313 (+ 210us) server_negotiation.cc:407] Connection header received
0809 23:18:50.262401 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.262401 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.262410 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.262436 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.262728 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45054: BlockingRecv error: recv got EOF from 127.0.0.1:45054 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 23:18:50.272413 9056 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:50.271659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45066 (local address 127.2.76.253:44163)
0809 23:18:50.271688 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.271689 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.271893 (+ 204us) server_negotiation.cc:407] Connection header received
0809 23:18:50.271976 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.271977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.271985 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.272010 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.272377 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45066: BlockingRecv error: recv got EOF from 127.0.0.1:45066 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 23:18:50.290086 9056 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:50.289402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45076 (local address 127.2.76.253:44163)
0809 23:18:50.289435 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.289437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.289560 (+ 123us) server_negotiation.cc:407] Connection header received
0809 23:18:50.289590 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.289590 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.289600 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.289669 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.290043 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45076: BlockingRecv error: recv got EOF from 127.0.0.1:45076 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 23:18:50.323585 9056 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:50.322928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45092 (local address 127.2.76.253:44163)
0809 23:18:50.322964 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.322966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.323079 (+ 113us) server_negotiation.cc:407] Connection header received
0809 23:18:50.323123 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.323125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.323135 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.323197 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.323538 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45092: BlockingRecv error: recv got EOF from 127.0.0.1:45092 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 23:18:50.389281 9056 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:50.388560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45104 (local address 127.2.76.253:44163)
0809 23:18:50.388624 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.388626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.388725 (+ 99us) server_negotiation.cc:407] Connection header received
0809 23:18:50.388795 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.388796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.388807 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.388881 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.389211 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45104: BlockingRecv error: recv got EOF from 127.0.0.1:45104 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 23:18:50.519191 9056 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:50.518474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45118 (local address 127.2.76.253:44163)
0809 23:18:50.518551 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.518553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.518651 (+ 98us) server_negotiation.cc:407] Connection header received
0809 23:18:50.518750 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.518751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.518763 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.518828 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.519149 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45118: BlockingRecv error: recv got EOF from 127.0.0.1:45118 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 23:18:50.776939 9056 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:18:50.776374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45132 (local address 127.2.76.253:44163)
0809 23:18:50.776426 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:50.776428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:50.776462 (+ 34us) server_negotiation.cc:407] Connection header received
0809 23:18:50.776483 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:50.776483 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:50.776499 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:50.776537 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:18:50.776879 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45132: BlockingRecv error: recv got EOF from 127.0.0.1:45132 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 23:18:51.127076 9215 catalog_manager.cc:1259] Loaded cluster ID: 9e4a9ec0f1404709883f3f22df5307da
I20260809 23:18:51.127134 9215 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da: loading cluster ID for follower catalog manager: success
I20260809 23:18:51.127775 9215 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da: acquiring CA information for follower catalog manager: success
I20260809 23:18:51.128011 9215 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 969f3bdda38a41ceb298a21a54c5e4da: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:18:51.215516 9286 catalog_manager.cc:1259] Loaded cluster ID: 9e4a9ec0f1404709883f3f22df5307da
I20260809 23:18:51.215581 9286 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3: loading cluster ID for follower catalog manager: success
I20260809 23:18:51.216298 9286 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3: acquiring CA information for follower catalog manager: success
I20260809 23:18:51.216560 9286 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 500705a149e64dc6b463416e1c148ac3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:18:51.292779 9422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:51.290180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45136 (local address 127.2.76.253:44163)
0809 23:18:51.290303 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:51.290306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:51.290362 (+ 56us) server_negotiation.cc:407] Connection header received
0809 23:18:51.290455 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:51.290456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:51.290512 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:51.290598 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:51.290763 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:51.291262 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:51.291962 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:51.292188 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:51.292257 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:51.292435 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:51.292556 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:51.292633 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:51.292676 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 23:18:51.292694 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":53,"threads_started":1}
I20260809 23:18:51.296721 9438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:51.294437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45014 (local address 127.2.76.252:39377)
0809 23:18:51.294529 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:51.294531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:51.294540 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:18:51.294604 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:51.294605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:51.294647 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:51.294729 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:51.294811 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:51.295230 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:51.296008 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:51.296214 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:51.296278 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:51.296407 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:51.296504 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:51.296579 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:51.296623 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 23:18:51.296646 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70,"thread_start_us":41,"threads_started":1}
I20260809 23:18:51.300513 9438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:51.298057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45028 (local address 127.2.76.252:39377)
0809 23:18:51.298092 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:51.298093 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:51.298208 (+ 115us) server_negotiation.cc:407] Connection header received
0809 23:18:51.298258 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:51.298259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:51.298267 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:51.298359 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:51.298487 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:51.299117 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:51.299864 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:51.300058 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:51.300183 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:51.300310 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:51.300334 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:51.300377 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:51.300424 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 23:18:51.300440 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:18:51.300973 9446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:51.298023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47806 (local address 127.2.76.254:39259)
0809 23:18:51.298141 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:51.298143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:51.298370 (+ 227us) server_negotiation.cc:407] Connection header received
0809 23:18:51.298422 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:51.298423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:51.298466 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:51.298537 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:51.298677 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:51.299112 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:51.300131 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:51.300435 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:51.300486 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:51.300689 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:51.300765 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:51.300826 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:51.300881 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 23:18:51.300901 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":43,"threads_started":1}
I20260809 23:18:51.301131 9422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:18:51.298274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45142 (local address 127.2.76.253:44163)
0809 23:18:51.298324 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:18:51.298325 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:18:51.298369 (+ 44us) server_negotiation.cc:407] Connection header received
0809 23:18:51.298444 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:18:51.298444 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:18:51.298454 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:18:51.298495 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:18:51.298634 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:51.299203 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:51.300150 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:18:51.300416 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:18:51.300450 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:18:51.300855 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:18:51.300884 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:18:51.300906 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0809 23:18:51.301050 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 23:18:51.301074 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 23:18:51.302579 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 8771
I20260809 23:18:51.313386 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 8918
I20260809 23:18:51.325706 2355 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvJxnr9/build/release/bin/kudu with pid 9059
2026-08-09T23:18:51Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2838 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5775 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (293033 ms total)
[ PASSED ] 54 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
2 FAILED TESTS